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Preface 


The  purpose  of  this  thesis  was  to  identify  or  develop  an  appropriate  method  for 
evaluating  the  effectiveness  of  a  unit-level  Air  Combat  Mission  Planning  (ACMP) 
process.  Our  sponsors  at  Headquarters,  Air  Combat  Command  (HQ  ACC)  and  Air  Force 
Material  Command's  Electronic  Systems  Center  (ESC)  are  upgrading  mission  planning 
capabilities  that  are  critical  for  overall  combat  effectiveness.  We  sought  to  provide  HQ 
ACC  and  ESC  a  tool  for  designing  the  best  possible  unit-level  mission  planning  process. 

Our  work  was  extensive  and  comprehensive.  Our  detailed  study  of  the  ACMP 
process  and  review  of  Desert  Storm  mission  planning  lessons  learned  formed  the 
foundation  for  a  conceptual  model  of  the  generic  ACMP  process  that  we  validated.  The 
conceptual  model  provides  a  starting  point  for  the  ACMP  Evaluation  Methodology  that 
we  developed  and  demonstrate  in  this  thesis.  From  our  research,  we  draw  conclusions 
about  the  Air  Force’s  ability  to  evaluate  and  improve  the  unit-level  ACMP  process. 

Our  project  could  not  have  proceeded  without  the  invaluable  help  of  others.  Lt 
Col  Art  Michel  from  HQ  ACC  and  Lt  Col  Tony  Sharon  from  ESC  not  only  sponsored  the 
project,  they  were  always  available  to  provide  information,  guidance,  and  assistance. 
Their  staffs,  especially  those  who  helped  with  conceptual  model  development  and 
validation,  never  hesitated  to  make  sure  we  truly  understood  the  difficulties  aircrews  face. 
We  are  deeply  indebted  to  Dr.  David  K.  Vaughan,  Maj  Gordon  D.  Wishon,  and  Capt 
Michael  A.  Morabito,  our  faculty  advisors,  who  spent  many  hours  assisting  us. 

Finally,  we  want  to  thank  our  families  for  all  their  help.  This  thesis  could  not 
have  been  completed  without  the  understanding  and  patience  of  Dawn,  Michael,  .^nd 
Matthew  Gaylord. 


Thomas  L.  Gaylord 
George  L.  Sowell 
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Abstract 

This  study  modeled  the  USAF  unit-level  Air  Combat  Mission  Planning  (ACMP) 
process  and  developed  a  methodology  for  evaluating  the  efTectiveness  of  changes  to  the 
process.  The  study  also  reports  relevant  mission  planning  lessons  learned  from  Desert 
Storm  and  demonstrates  application  of  the  developed  evaluation  methodology.  An 
exploratory  multidisciplinary  literature  review  builds  a  research  foundation  for  the 
evaluation  methodology.  Foundation  areas  reviewed  include  unit-level  operations, 
command  and  control,  planning,  decision  theory,  decision  aiding,  and  model-based 
problem  solving.  The  conceptual  model  of  the  planning  process  defines  sixteen  functions 
starting  from  receipt  of  tasking  and  concluding  with  execution  departure,  For  each  of  the 
sixteen  functions,  objectives,  decisions,  data  requirements,  and  products  are  defined. 
Within  the  developed  model-based  evaluation  methodology,  the  conceptual  model  eases 
development  of  specific  models  that  are  used  to  evaluate  effectiveness  of  changes  to  the 
mission  planning  process.  The  methodology  separately  evaluates  mission  and  behavioral 
vectors  of  plan  quality.  The  demonstration  applied  the  methodology  to  an  Air  Combat 
Command  requirements  evaluation  management  issue,  and  confirmed  the  reasonableness 
of  the  ACMP  evaluation  methodology.  The  study  highlights  the  difficulty  of  evaluating 
the  effects  of  behavioral  factors  and  recommends  further  research. 
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A  MOD£L>BAS£D  M£THODOLOGY  FOR 
£VALUATING  THE  UNIT-LEVEL 
AIR  COMBAT  MISSION  PLANNING  (ACMP)  PROCESS 


I.  Introduction 


General  Issue 

Unit-level  mission  planning  is  excessively  time  consuming  and  labor  intensive.  It 
involves  the  collection  and  integration  of  many  pieces  of  information  (i.e.,  target  data, 
aircraft  performance  data,  threat  data,  current  and  forecast  weather,  friendly  and  enemy 
force  structure)  from  many  distributed  data  sources  (Bahnij,  1985:1-3).  Mission  planners 
must  accurately  assemble  these  pieces  of  information  into  an  accunite,  comprehensive 
plan  that  achieves  the  mission  objectives  and  tasking  while  trying  to  maximize  mission 
effectiveness  and  survivability  of  the  aircrew.  I'he  criticality  of  automating  our  ability  to 
manage  this  planning  complexity  was  highlighted  by  Desert  Storm's  chief  air  war 
planner,  Maj  Gen  Buster  Gtosson,  who  discussed  the  effects  of  increasing  weapons 
sophistication  upon  planning:  “without  the  aid  of  computer  mission  planning,  you  won't 
have  the  capacity  to  take  advantage  of  the  capabilities  that  the  taxpayer  paid  for.  You 
must  have  the  aid  of  computer  mission  planning”  (Capaccio,  1991:3). 

As  the  new  Air  Combat  Command  (ACC)  considers  the  increasingly  important 
need  for  mission  planning,  it  is  useful  to  understand  the  changing  role  of  unit-level 
mission  planning  in  ACC's  predecessors.  In  the  Tactical  Air  Forces  (TAP),  mission 
planning  was  part  of  the  planning  and  execution  phase  of  the  tactical  Command.  Control, 
Commiinicatious.  and  Intelligence  (C^I)  cycle.  More  specifically,  it  has  been  a  subset 
function  of  the  Tactical  Air  Control  System  (TACS)  (Teas,  1989:1-4).  Automation  of 
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TAGS  has  been  discussed  since  the  early  1960s  (Lindsay  and  Morris.  1982:227).  More 
recently,  a  number  of  systems  attempted  to  automate  various  aspects  of  TAGS  to  include 
the  mission  planning  process.  A  problem  that  has  plagued  the  TAGS  and  mission 
planning  during  this  time  is  that  an  increasing  number  of  independently-developed, 
“stove-pipe”  systems  have  been  fielded  without  adequate  integration.  This  problem  was 
so  severe  that  it  prompted  the  formation  of  a  General  Officer's  Steering  Group  (GOSG) 
for  Tactical  Battle  Management  (TBM)  in  1987  (Teas,  1989:10). 

During  operation  Desert  Storm,  the  squadron  mission  planning  process  used  by 
U.S.  aircrews  was  exercised  during  large  scale  combat  conditions  for  the  first  time  since 
the  Vietnam  conflict.  A  number  of  problems  with  the  planning  process  surfaced  as  a 
result  of  operation  Desert  Storm.  Problems  disseminating  and  breaking  down  the  Air 
Tasking  Order  (ATO),  problems  receiving  intelligence  information,  and  less  than 
adequate  training  on  planning  aids  are  a  few  of  those  problems  frequently  cited.  Many 
others  are  documented  in  after-action  reports  (Cosby,  1991:5).  In  light  of  the  problems 
surfaced  by  Desert  Storm  and  the  acknowledged  “stove-pipe”  problem,  the  using 
commands  are  implementing  changes  in  the  mission  planning  process  and  to  the 
computers  that  the  aircrews  use  to  plan  missions  (HQ  TAC/DR,  1991:5).  Additionally, 
Air  Force  Material  Command  (AFMC)  is  busy  developing  the  next  generation  of 
computer  systems  to  assist  aircrew  planning  for  our  current  and  future  weapon  systems 
(Hughes,  1991:53). 

The  problem  facing  the  Air  Force  today  is  that  the  operators  and  developers 
involved  in  the  process  do  not  know  which  changes  will  provide  the  greatest 
improvement  in  the  overall  process.  Gen  Robert  Russ,  Tactical  Air  Command  (TAC) 
Commander  in  early  1991,  pointed  out  the  need  to  keep  an  open  mind  about  what  a  new 
automated  system  will  do  instead  of  just  automating  the  manual  process  that  has  been  in 
use  for  forty  years.  The  program  manager  for  acquiring  the  next  generation  of  systems 
characterized  this  open-minded  approach  as  a  new  way  of  doing  business  witli  an  intense 
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focus  on  how  to  set  up  and  define  the  requirements  (Hughes.  1991:53).  While  previous 
attempts  at  automation  have  aimed  to  increase  the  efficiency  of  various  components  of 
the  “manual  process,”  a  systematic  approach  to  process  improvement  is  needed.  This 
issue  is  important  because  the  effectiveness  of  our  combat  air  operations  is  directly  linked 
to  the  effectiveness  of  the  squadron  mission  planning  process.  Lt  Gen  Thomas 
McMullen,  then  TAG  Vice-Commander,  summarized  the  need  for  mission  planning  and 
other  C^l  elements  in  a  presentation  at  National  Defense  University:  “We  have  to  have  a 
good  plan,  good  information,  good  ability  to  control  the  force  to  get  them  where  we  want 
them,  and  then  we  have  to  be  able  to  sense  what's  going  on  and  adjust.  C^l  really  is  the 
sum  of  the  things  done  to  achieve  proper,  effective  employment  of  tactical  air”  (Coakley. 
1991:19). 

Specific  Problem 

The  Air  Force  has  established  a  requirement  for  automation  of  the  unit-level 
mission  planning  process,  and  AFMC  has  established  a  program  to  meet  that  need  which 
recognizes  the  need  to  change  the  planning  process,  not  just  automate  it.  The  Air  Force 
wants  the  best  iwssible  automated  planning  process  (SAF/AQ,  1992: 1-3;  Hughes, 
1991:53).  It  cannot  be  assumed  that  automating  the  planning  process  will  improve  it.  In 
fact,  rigorous  evaluations  are  required  to  show  that  a  newly  automated  process  is  superior 
to  the  previous  process  (Hopple,  1^6:949).  It  logically  follows  that  methods  for 
conducting  rigourous  evaluation  of  proposed  changes  to  unit-level  mission  planning 
processes  are  needed  to  obtain  the  best  possible  automated  planning  process.  We  assume, 
based  on  the  requirement  to  obtain  the  best  possible  process  and  our  experience  with 
ACMP  changes,  that  ACMP  changes  should  maintain  or  increase  overall  planning 
effectiveness.  Currently,  no  method  exists  for  evaluating  the  effectiveness  of  unit-level 
mission  planning  process.  The  goal  of  this  research  project  is  to  identify  or  develop  an 
appropriate  method  for  evaluating  the  effectiveness  of  a  unit-level  missi'^n  planning 
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process.  As  part  of  accomplishing  this  goal,  a  descriptive  model  of  a  generic  unit-level 
mission  planning  process  will  be  developed  to  facilitate  understanding  of  the  process 
being  evaluated.  If  successful,  this  model  and  evaluation  method  will  help  developers 
and  users  of  automated  unit-level  mission  planning  systems  to  design  the  best  possible 
unit-level  mission  planning  process. 

Scope 

In  this  project,  the  unit-level  mission  planning  process  is  defined  to  include  those 
functions,  decisions,  and  activities  that  occur  within  an  operational  air  combat  wing  and 
squadron  that  transform  the  mission  tasks  found  in  the  Air  Tasking  Order  (ATO)  into 
individual  flight  plans,  data,  and  materials  used  by  the  individual  aircrews.  Furthermore, 
we  are  specifically  concentrating  on  the  “wartime”  mission  planning  process  as  opposed 
to  one  that  might  be  used  during  training  or  peacetime  flying.  This  wartime  emphasis 
was  chosen  because  aircrews  have  tended  not  to  plan  in  peacetime  using  the  processes 
they  arc  required  to  use  in  combat  (HQ  TAC/IX),  1991:Al-28).  It  is  worthwhile  to 
capture  as  much  knowledge  as  possible  on  how  our  processes  function  under  real  combat 
situations. 

A  further  limitation  on  the  scope  of  our  effort  will  be  the  level  at  which  we 
describe  and  model  the  planning  process.  Our  intent  is  to  model  a  conceptual  mission 
planning  process,  primarily  because  there  is  no  single  definitive  “mission  planning 
process”  that  is  used  by  all  air  combat  units.  Each  wing  or  squadron  may  plan  differently 
according  to  the  missions  that  it  performs  (Bahnij,  1985:11-14).  This  conceptual  approach 
is  supported  by  others  in  the  Command  and  Control  research  field: 

Both  the  complexity  of  the  role  and  the  changing  nature  of  the  situations  that 
military  systems  are  expected  to  cope  with  ensure  that  any  attempts  to  model 
them  are  bound  to  lead  to  frustration  if  a  'definitive'  model  is  sought. 

(Moynihan  and  Bowen,  1987:284) 
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In  addition  to  excluding  planning  for  training  missions,  we  will  exclude  from  our 
research  focus  the  planning  of  Single  Integrated  Operational  Plan  (SlOP)  nuclear 
missions.  Special  Operations  Forces  (SOF)  missions,  and  missions  conducted  on  highly 
classified,  “black”  systems.  SIOP,  SOF,  and  “black”  planning  processes  diverge  from 
routine  unit-level  processes,  and  their  inclusion  would  severely  complicate  the  research. 

During  the  conduct  of  this  research  the  Air  Force  was  undergoing  a  dramatic 
restructuring  of  commands.  Air  Force  Systems  Command  (AFSC)  and  Air  Force 
lx>gistics  Command  merged  into  Air  Force  Material  Command  (AFMC).  Combat 
components  of  Strategic  Air  Command  merged  with  Tactical  Air  Command  components 
to  form  Air  Combat  Command  (ACC).  Our  research  was  conducted  to  be  relevant  to  this 
new  command  structure.  However,  the  timing  of  our  research  required  that  we  draw  data 
and  conclusions  from  work  with  the  new  conunands*  predecessors  in  AFSC  and  TAC. 
Our  research  prior  to  the  mergers  was  facilitated  by  decisions  implemented  prior  to  the 
command  restructuring.  AFSC  formed  a  single  program  office  at  its  Electronic  Systems 
Division  which  was  responsible  for  all  unit-level  mission  planning  system  acquisition, 
and  the  operational  commands  created  a  single  office  at  TAC  that  was  primarily 
responsible  for  coordinating  all  Air  Force  mission  planning  requirements.  We  were 
sponsored  by  and  worked  closely  with  these  two  consolidated  offices.  These 
consolidated  offices  continue  to  have  primary  responsibility'  for  Air  Force  mission 
planning  acquisitions  and  requirements  in  their  new  commands,  ACC  and  AFMC. 
Therefore,  we  confidently  used  data  from  AFSC  and  TAC  to  support  our  analysis  of  the 
ACMP  processes  in  ACC. 

Definition  of  Terms 

Three  key  conceptual  areas  form  the  foundation  for  our  research:  1)  Command 
and  Control  (C^),  2)  Planning,  and  3)  Decision  Thsory  and  Decision-Aiding.  They  form 
a  hierarchy  of  concepts  important  to  understanding  the  unit-level  mission  planning 
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prcx;ess.  Each  term  is  introduced  in  this  section  and  further  discussed  in  the  literature 
review.  A  fourth  conceputal  area,  modeling,  provides  the  foundation  for  our  research 
methodologies.  It  is  also  introduced  in  this  section  and  then  is  further  defined  in  Chapter 
III,  Methodology. 

Command,  Control.  Communications,  and  Intelligence  (C^I)  can  be  summarized 
as  *‘what  melds  the  ‘stuff  of  defense  together  into  an  effective  fighting  machine*’ 
(Coakley,  1991:xvi).  Command  is  receiving  information,  making  decisions  based  on  that 
information,  and  sending  orders.  Control  is  the  “lines  of  command"  or  the  authority 
exercised  by  a  commander.  Communications  is  the  means  of  transmission  of 
information,  and  Intelligence  is  the  collection,  analysis  and  presentation  of  information 
about  an  enemy,  potential  enemy,  or  the  commander's  own  forces  (Coakley,  1991  :xvi- 
xix).  Command  and  Control  (C^)  is  often  used  synonymously  with  C^I,  although  it 
focuses  less  on  the  technical  issues  involved  in  communication  and  intelligence  (Wohl, 
1991: 1 1).  Planning,  the  central  issue  of  this  paper,  is  an  integral  element  of  C^  as  it  is 
used  to  shift  decisions  to  periods  of  reduced  activity  (Weissinger-Baylon,  1986:40). 

Planning  is  a  process  which  can  be  difficult  to  define.  “Planning  can  be 
considered  the  process  of  solving  a  problem  in  a  real  world  environment  where  only 
incomplete  or  inexact  information  exists,  and  the  planner  cannot  completely  know  the 
consequence  of  a  given  action”  (Leroux  and  Li.  1990:64).  Planning  is  a  continuous 
process,  while  plans  reflect  actions  to  be  “taken  during  a  prescribed  period  of  time  to 
achieve  specific  objectives”  (Branch,  1983:134).  Despite  differing  definitions,  planning 
has  five  basic  steps: 

1 )  collection  of  descriptive  and  prescriptive  information,  2)  analysis  of  this 
information,  3)  reaching  conclusions  and  making  decisions,  4)  designation  and 
effectuation  of  implementing  actions,  and  5)  comparison  of  the  results  of 
planning  witli  its  intentions,  and  corresponding  adjustment  or  revision  as 
indicated.  (Branch,  1983:vii) 
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Planning  and  decision-making  are  interrelated.  They  both  serve  the  functions  of  choice 
and  making  progress  as  i’’  .*stratcd  in  Figure  1-1.  Many  theories  attempt  to  describe 
decision-making.  Most  of  these  theories  assume  that  choices  are  made  intentionally 
based  on  expectations  about  the  consequences  of  current  action.  Within  the  theories,  four 
elements  are  assumed:  1)  knowledge  of  alternatives,  2)  knowledge  of  consequences,  3)  a 
consistent  preference  ordering,  and  4)  a  decision  rule  (March  and  Wsissinger-Baylon, 
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Figure  1>1.  A  View  of  Planning  Realities  (Friend  and  Hkkling,  1987:4) 
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1986: 12-13).  While  decision-making  is  only  one  of  five  of  the  planning  steps  listed 
above,  it  is  perhaps  the  most  critical  element  for  measuring  effectiveness  of  planning. 
Decision  theory  and  decision-aiding  will  be  further  explored  in  the  literature  review. 

Models  are  abstractions  or  descriptions  of  an  actual  or  proposed  system  (Pritsker, 
1986:4).  The  system,  idea,  situation,  policy,  or  phenomena  to  be  modeled  is  called  the 
problem  entity  (Sargent,  1991:38).  Models  can  range  from  an  exact  representation  of  the 
problem  entity,  such  as  a  scale  or  full  physical  model,  to  a  very  abstract  representation, 
such  as  a  mathematical  formula.  A  conceptual  model  is  defined  by  Sargent  as  a 
""mathematical/logical/verbal  representation  (mimic)  of  the  problem  entity  developed  f  or 
a  particular  study"  (38).  This  classification  by  Sargent  of  a  conceptual  model  is  in 
contrast  with  that  of  the  computerized  model  that  he  defines  as  “the  conceptualized  model 
implemented  on  a  computer"  (38).  We  use  conceptual  and  computerized  models  to  fulfill 
our  research  objectives. 

Objectives 

As  introduced  in  the  earlier  section  on  the  specific  problem,  our  research  problem 
was  to  identify  or  develop  an  appropriate  method  that  can  be  used  to  evaluate  a  unit-level 
air  combat  mission  planning  process.  Given  our  resulting  project  goal  of  defining  a 
model  of  the  generic  mission  planning  process  and  identifying  or  developing  an 
appropriate  evaluation  methodology,  we  established  three  related  objectives. 

Research  Objective  #1.  Define  a  conceptual  or  “generic”  Air  Combat  Mission 
Planning  (ACMP)  process  and  select,  or  adapt,  an  existing  C3  or  combat  planning  model 
that  accurately  reflects  the  defined  ACMP  process. 

Research  Objective  #2.  Derive  from  previous  research  a  general  method  for 
evaluating  the  process  effectiveness  of  an  ACMP  process. 
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Research  Objective  #3.  Demonstrate  application  of  the  conceptual  ACMP  model 
resulting  from  Research  Objective  #1,  and  the  general  ACMP  process  evaluation  method 
developed  in  Research  Objective  #2,  to  the  investigation  of  a  specific  ACMP 
management  issue. 

The  primary  elements  of  our  research  are  summarized  in  Figure  1-2. 


PROBLEM 

OBJECTIVES 


RESULT 


Figure  1-2.  Research  Sununary 

Overview  of  the  Thesis 

Our  literature  review.  Chapter  II,  is  divided  into  three  sections.  The  first  section, 
a  walkthrough  of  the  Air  Combat  Mission-Planning  (ACMP)  process,  introduces  the 
reader  to  the  mission  planning  process  by  identifying  the  key  activities  and  players  in  the 
process.  We  collected  this  information  to  build  our  conceptual  model  for  objective  #1. 
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The  second  section  of  our  literature  review  presents  a  summary  ot'  mission 
planning  lessons  learned  from  Desert  Storm.  The  combination  of  the  walkthrough  of  the 
ACMP  process  and  the  lessons  learned  constitutes  a  review  of  applicable  operational 
literature.  We  analyzed  this  operational  review  and  developed  seven  mission  planning 
trends  which  are  presented  at  the  end  of  section  two.  Developing  this  operational  review 
and  trends  improved  our  understanding  of  the  process  prior  to  building  our  conceptual 
model.  The  lessons  and  trends  further  provided  an  insight  into  future  operational 
evaluation  requirements  that  helped  us  develop  our  methodology  for  objective  #2. 

The  third  section  of  our  literature  review  presents,  and  then  summarizes,  academic 
research  of  topics  related  to  our  second  research  objective  of  deriving  a  general  method 
for  evaluating  process  effectiveness:  1)  Command  and  Control  (C^).  2)  Planning,  and  3) 
Decision  Theory  and  Decision-Aiding.  We  discovered  after  completing  this  academic 
review  that  parallels  exist  between  the  academic  findings  and  the  mission  planning 
trends.  These  parallels  are  presented  at  the  end  of  Chapter  II  in  our  literature  review 
analysis/conclusions. 

Our  methodology  description.  Chapter  III,  begins  with  a  literature  review  of  two 
topics  that  are  integral  to  our  research  effort;  1)  modeling  and  2)  model-based  problem 
solving.  Following  this  review,  we  discuss  and  justify  the  approach  taken  to  achieve  each 
research  objective.  For  research  objective  #1,  we  describe  each  step  of  the  model 
building  process  used  to  develop  and  validate  our  conceptual  ACMP  process  model.  For 
research  objective  #2,  we  identify  the  primary  research  that  influenced  our  thoughts  and 
discuss  factors  we  considered  while  developing  our  evaluation  methodology,  For 
research  objective  #3,  we  address  two  aspects  of  our  method  demonstration.  First,  we 
describe  the  steps  taken  to  conduct  the  evaluation  and  second,  we  define  the  specific 
management  issue  and  mission  pluming  scenario  used  in  the  evaluation. 
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In  Chapter  IV,  we  present  the  results  of  each  research  objective.  For  objective  #1 
we  present  our  conceptual  model  and  validation  results.  The  conceptual  model  is 
composed  of  two  parts:  1)  a  short  narative  which  identifies  the  purpose,  scope,  level  of 
detail,  boundaries,  assumptions,  and  data  categories  of  the  model;  and  2)  graphical 
representations  wliich  illustrate  the  relationships,  objectives,  decisions,  and  data 
requirements  for  each  ACMP  function.  After  presenting  our  conceptual  model,  we 
introduce  and  discuss  the  evaluation  methodology  we  developed  for  research  objective 
#2.  Chapter  IV  concludes  by  presenting  the  results  of  research  objective  ^3.  The  objective 
#3  results  include  not  only  the  simulation  results  that  determined  the  proposed  solution  to 
our  demonstration  management  issue,  but  also  document  each  step  taken  to  conduct  the 
evaluation  and  produce  the  simulation  results. 

In  Chapter  V  we  summarize  our  research  by  drawing  conclusions  about  our 
ability  to  achieve  our  research  goal.  We  highlight  the  strengths  and  limitations  of  our 
evaluation  methodology  and  discuss  their  importance.  Finally,  we  conclude  our  thesis 
with  recommendations  for  further  research  on  Air  Combat  Mission  Planning. 
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II.  Literature  Rftview 

Introduction  of  Tonics 

Our  literature  review  spans  three  major  areas  of  importance  to  our  research:  the 
air  combat  mission  planning  process,  mission  planning  lessons  learned  from  Desert 
Storm,  and  academic  literature  relevant  to  our  topic.  Because  our  search  for  appropriate 
methodologies  of  evaluating  the  Air  Combat  Mission  Planning  process  involved  many 
academic  areas,  our  literature  review  was  exploratoiry.  While  we  found  no  prior 
research  on  our  specific  topic,  we  found  significant  research  that  was  applied  to 
strucmring  our  effort.  Specifically,  the  literanire  of  the  ACMP  process  provides  a 
foundation  for  Objective  #1  ’s  conceptual  model  of  the  ACMP  process.  The  lessons 
learned  from  Desert  Storm  provide  specific  data  on  wartime  unit-level  planning 
operations  for  the  conceptual  model  and  provide  departure  points  for  considering 
operational  evaluation  requirements  for  Objective  #2*s  evaluation  methodology,  Our 
review  of  academic  literature-journals,  texts,  and  technical  reports -provides  a  broad 
review  of  issues  relevant  to  derivation  of  an  ACMP  evaluation  methodology  and 
demonstration  of  that  methodology  in  Objective  #3.  Our  literature  review  summarizes 
the  ACMP  process  and  previous  research  relevant  to  evaluating  the  effectiveness  of  the 
ACMP  process. 

A.  WalkttLrou2h  of  the  Air  Combat  Mission-Planning  (ACMP)  Process 

The  daily  unit-level  mission  planning  process  begins  when  the  Air  Tasking  Order 
(ATO)  is  received  at  the  Wing  (Bahnij,  1985:11-4;  ESD/AVDB  and  MITRE  Corp., 
1991:35),  The  ATO  is  a  standardized  message  designed  for  tasking  air  units  to  carry  out 
daily  war  plans.  Specifically,  it  is  message  A658  in  the  United  States  Message  Text 
Format  (USMTF)  series  for  joint  service  communications  (Permenter,  1991:7).  The  ATO 
is  generated  at  the  theater  or  Tactical  Air  Control  Center  (TACC)  level  and  transmitted 
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electronically  to  all  flying  units  over  which  the  theater  or  Air  Component  Commander 
(ACC)  has  authority. 

During  Desert  Storm: 

the  ATO  grew  to  several  hundred  pages  [600  at  the  height  of  the  war]  and 
served  as  the  single  source  document  for  the  entire  flying  operation  of  Desert 
Storm.  It  was  a  48-hour  process,  so  the  initial  planning  was  for  targets  to  be 
struck  two  days  down  the  road.  It  was  a  never-ending  cycle,  continuously 
taking  place  for  as  long  as  air  operations  were  necessary.  (Homer,  1991:22) 

The  ATO  assigns  various  units  with  specific  missions  to  be  carried  out  during  the 
next  24-hour  period.  Information  contained  in  the  ATO  includes: 

target  identification  and  location,  time  the  bombs  need  to  be  on  target  (TOT), 
unit  assigned,  type  and  number  of  aircraft,  aircraft  weapons  configuration,  the 
type,  number  and  call  sign  of  all  other  support  players  (such  as  reconnaissance, 
air  escort,  defense  suppression  (Wild  Weasels),  tankers,  AWACS,  electronic 
Jammers,  Army,  Navy,  Marines).  (Bahnij,  191^:11-4) 

• 

The  ATO  is  transmitted  by  secure  metuis  from  the  TACC  to  the  base  command 
post  where  it  is  deciphered  and  reviewed  by  the  Wing  Battle  Staff.  After  this  cursory 
review,  the  ATO  is  hand  carried  to  the  wing  “Frag  Shop,”  where  it  is  broken  down  by 
mission  number.  “The  first  action  of  the  Frag  Shop  is  to  determine  if  there  are  enough 
aircrews,  munitions  and  aircraft  to  meet  the  wing's  tasking”  (ESD/AVDB  and  MITRE 
Corp.,  1991:35).  The  Frag  Shop  mission  planner(s)  also  coordinate  with  other  wing  staff 
members  from  weather  and  intelligence  and  with  the  individual  squadrons  if  required. 
After  determining  the  wing's  capability  to  support  the  tasking,  the  Frag  Shop  Officer 
briefs  the  Wing  Commander  with  his  recommendations  about  which  missions  to  accept 
or  reject.  After  a  decision  from  the  Wing  Commander,  the  Frag  Shop  notifies  the  TACC 
on  the  ability  of  the  Wing  to  support  the  tasked  missions. 

After  the  decision  by  the  Wing  Commander,  the  Frag  Shop  further  refines  the 
assigned  mission  tasking,  divides  the  mission  sorties  among  the  Wing's  squadrons,  and 
prepares  the  information  into  relevant  format  for  release  to  the  squadrons.  Any  tasking 
changes  that  may  occur  from  the  TACC  are  updated  by  the  Frag  Shop  as  they  are 
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received.  Upon  final  review  by  the  Wing  Commander,  the  day's  schedule,  usually  called 
the  Frag,  is  completed  and  released  to  the  squadrons.  Mission  Planning  Cell(s),  andyor 
Mission  Commanders  for  mission  or  individual  flight  planning  (ESD/AVDB  and  MITRE 
Corp„  1991:36). 

In  some  wings,  an  organization  called  the  Mission  Planning  Cell  (MPC)  is  used  to 
help  plan  and  coordinate  the  larger  strike  packages.  These  MPCs  can  be  organized  at 
either  the  Wing  level  or  Squadron  level,  depending  upon  the  missions  or  mission  types 
flown.  In  addition,  a  Mission  Commander  is  often  assigned  to  coordinate  with  the 
various  packages  or  flights  of  aircraft  assigned  to  a  particular  mission.  These  Mission 
Commanders  perform  a  key  role  and  are  assisted  in  the  planning  by  the  flight  leaders 
(Permenter,  1991:7). 

Upon  receipt  of  the  “Frag’’at  the  Mission  Planning  Cell  or  squadron,  a  mass 
briefing  is  normally  held  to  “inform  selected  pilots  the  high  level  details  of  the  unit's 
tasking”(BahniJ,  1985:11-5).  These  selected  pilots,  usually  the  mission  commanders  or 
flight  leaders,  receive  current  intelligence  briefings,  war  status,  weather  briefings  for  the 
local,  cnroute  and  target  areas,  and  specific  details  about  the  day's  missions  and  taskings. 
After  the  briefings  are  concluded,  the  mission  commanders,  along  with  the  flight  leaders, 
begin  assembling  their  specific  mission  plans.  Concerns  at  this  level  are  interface  issues 
with  support  elements,  coordinating  mutual  support  among  flights  (the  lowest  mission 
element;,  and  deconflicting  time  and  space  requirements  (Bahnij,  1985:11-6). 

After  the  mission  commanders  and/or  flight  leaders  have  agreed  on  the  top-level 
details,  individual  flight  planning  for  each  airciaft  or  flight  of  aircraft  begins.  This 
individual  or  flight  mission  planning  continues  throughout  the  day  as  specific  missions 
are  executed.  It  is  important  to  note  here  that  this  planning  activity  is  a  continuous  cycle. 
Missions  are  scheduled  throughout  the  day  and  preparation  times  for  individual  flights 
are  sometimes  short  due  to  the  time  it  takes  to  “shredout”  the  ATO,  coordinate  among 
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participants,  and  collect  planning  data.  Often,  individual  aircrew  members  are  left  only 
short  periods  of  time  to  actually  prepare  their  individual  flight  plans  and  materials  due  to 
the  time  taken  to  frag  the  ATO  and  coordinate  mission  requirements.  Available  time  for 
individual  aircrew  planning  can  be  as  long  as  several  hours  for  strategic  targets  or  as  short 
as  15  to  20  minutes  for  requirements  such  as  Battlefield  Air  Interdiction  (BAl) 
(Fermenter,  1991:58“69).  “The  time  left  for  individual  flight  mission  planning  is  less 
than  one  hour;  30  to  45  minutes  is  typical"  (Bahnij,  1985:11-10). 

After  receiving  initial  intelligence  and  weather  briefings,  the  aircrew  members 
performing  flight  planning  for  the  flight  of  aircraft  begin  to  collect  all  the  necessary 
materials  (e.g.,  maps,  forms.  Data  Transfer  Devices  (DTD),  and  notices  to  airmen 
(NOl  AMSl)  needed  to  prepare  a  flight  plan  and  load  the  aircraft.  In  addition,  they  may 
consult  any  one  of  several  aeronautical  charts  that  are  continuously  updated  with 
intelligence  information,  tanker  positions,  restricted  zones  or  other  important  information. 
After  collecting  this  information  and  materials,  the  aircrew  members  are  ready  to  begin 
laying  out  the  flight  plan.  Today,  several  computerized  or  automated  mission  planning 
systems  help  the  aircrew  member  formulate  and  prepare  his  individual  flight  plan.  Two 
systems  currently  in  service  in  most  fighter  squadrons  are  the  Mission  Support  System 
(MSS)  II,  a  dedicated  hardware  and  software  mission  planning  system,  and  FPLAN,  a 
less  capable,  yet  fast  and  user-friendly  software  package  that  runs  on  any  IBM- 
compatible  personal  computer.  Both  systems  have  the  capability  to  generate  a  flight  plan 
complete  with  fuel  calculations,  time,  distance,  heading,  altitudes,  navigation  points, 
communication,  and  weapons  information  that  is  required  for  all  flight  plans.  The 
calculations  and  forms  are  printed  out  at  the  users’  request. 

The  MSS  II  includes  many  additional  features  not  found  on  FPLAN.  These 
include  resident  digitized  aeronautical  charts,  wide  area  satellite  imagery,  expanded 
graphical  displays  of  threat  ranges,  and  decision  aids  for  threat  avoidance  and  route 
planning.  A  color  printer  is  provided  to  produce  strip  charts  and  maps  which  can  depict 
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the  flight  plan  or  threat  environment  and  can  be  taken  with  the  flight  forms  into  the 
cockpit.  This  ability  to  quickly  and  easily  produce  flight  materials,  or  Combat  Mission 
Folders  (CMF)  as  they  are  sometimes  called,  is  a  considerable  time-saver  when  compared 
to  manual  methods.  Every  bit  of  time  saved  by  not  having  to  draw  flight  plans  on  maps 
or  transfer  flight  plan  information  to  line-up  cards  or  Form  70s  is  time  the  pilot  or  aircrew 
member  can  spend  improving  the  plan  or  studying  the  plan  to  gain  “situational 
awareness”  (Bahnij:Il-14). 

Alsr ,  the  MSS  II  allows  the  aircrew  member  to  load  automatically  his  flight  plan 

information  into  a  Data  Transfer  Device  (DTD)  that  can  be  taken  to  the  aircraft.  This 

DTD  is  inserted  into  the  aircraft,  and  the  flight  plan  information  is  uploaded  into  the 

aircraft's  computers  and  navigation  systems.  By  allowing  the  computer  to  upload  the 

flight  plan  information,  pilots  eliminate  the  opportunity  for  entry  errors  and  create 
« 

additional  time  for  studying  the  mission. 

“By  the  time  all  these  distributed  duties  are  performed  and  the  initial  plan  is  made, 
there  is  little  or  no  time  left  for  replanning.  The  remaining  time  is  required  to  brief  the 
developed  plan  and  get  dressed  to  fly  the  mission”  (Bahnij,  1985:11-13).  A  flight  briefing 
is  held  among  the  flight  members  to  go  over  last-minute  execution  details.  After  the 
night  briefing  is  completed,  the  aircrew  member  is  ready  to  proceed  to  the  jet  and  begin 
pre-tlight  and  ground  checks.  After  the  aircraft  has  been  initialized  with  all  of  the  flight 
plan  information,  the  planning  process  ends,  and  the  execution  phase  begins. 

Mission  Planning  Lessons  Learned  from  Desert  Storm 

When  evaluating  a  system  or  process,  observing  or  testing  it  under  actual 
conditions  is  the  best  test  of  its  performance.  As  a  part  of  researching  our  first  objective 
to  build  a  conceptual  model  of  the  unit-level  mission  tanning  process,  we  studied  results 
from  Desert  Storm.  As  Secretary  of  Defense  Cheney  stated,  “we  need  to  look  ai 
Operations  Desert  Shield  and  Desert  Storm  for  indications  of  what  military  capabilities 
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we  may  need  not  just  in  the  next  ft  a  years,  but  in  10.  20  or  30  years  hence"  (Department 
of  Defense.  1 992: vi-vii).  Because  the  Air  Force  is  attempting  to  incorporate  lessons 
learned  from  Desert  Storm  into  future  changes,  these  lessons  are  directly  relevant  to  our 
research.  They  contributed  to  our  understanding  of  the  wartime  ACMP  process  modeled 
in  objective  ^1  and  influenced  our  operational  evaluation  requirements  researched  in 
objective  #2. 

Two  documents  provided  most  of  the  mission  planning  lessons  learned  presented 
in  this  section.  The  first  document  was  the  Department  of  Defense's  final  report  to 
Congress  on  the  conduct  of  the  Persian  Gulf  War.  This  document  addresses  the  events  of 
Desert  Shield/Desert  Storm  from  a  broad  military  perspective.  Although  broad  in  scope, 
the  document  sheds  light  on  many  problems  and  successes  in  areas  that  affected,  or  were 
affected  by,  unit-level  mission  planning.  The  second  document  is  a  technical  report 
published  by  the  MITRE  Corporation  under  contract  to  Electronic  Systems  Division 
(ESD)  for  HQ  TAC.  I'his  second  document  concentrates  heavily  on  the  specific  lessons 
learned  about  automated  mission  planning  systems.  HQ  TAC  collected  these  lessons 
during  a  symposium  in  April  1991. 

There  is  often  a  tendency  to  concentrate  on  negative  lessons.  However,  learning 
from  our  lessons  is  not  meant  to  detract  from  the  accomplishments  of  those  who 
supported  or  performed  the  thousands  of  hours  of  mission  planning  during  Desert  Storm. 
Although  we  highlight  the  problems,  one  should  keep  in  mind  there  were  many  more 
successes  than  failures.  These  successes  made  Operation  Desert  Storm  one  of  the  most 
successful  military  operations  in  history. 

We  tried  to  prest  t  lessons  that  were  supported  by  several  sources.  This  approach 
has  both  positive  and  negative  aspects.  On  one  hand,  it  al’  «  us  to  focus  on  major 
problems  that  apply  to  more  than  one  unit,  c.  approach  to  miss.jn  planning.  On  the  other 
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hand,  major  problems  or  opportunities  for  improvement  may  be  hidden  by  information 
thought  to  be  unimportant  and  not  reported  by  many  planners. 

After  sorting  through  the  source  documents,  mission  planning  lessons  were 
grouped  into  six  categories-  technology;  automation;  communication;  information  flow; 
organization  and  staffing;  and  training/proficiency.  Each  of  the  categories  is  discussed 
below. 

Technology.  The  DOD  report  to  congress  on  Desert  Storm  states  “The 
importance  of  technology  in  the  impressive  results  achieved  by  Coalition  air  operations 
will  be  given  special  prominence  as  strategists  assess  the  lessons  of  Desert  Storm’* 
(Department  of  Defense,  1992:xiii).  Secretary  of  Defense  Cheney  states  further  that  a 
“militaiy-technological  revolution  in  warfare”  was  demonstrated  during  the  war 
(Department  of  Defense,  1992:xii).  Improvements  in  technology  are  allowing  us  to 
communicate  faster,  obtain  more  information  about  the  environment,  and  increase  the 
tempo  of  war.  We  are,  in  effect,  changing  the  way  we  fight. 

Secretary  of  Defense  Cheney's  statement,  although  made  in  regard  to  all  aspects 
of  air  operations,  is  especially  valid  from  a  mission  planning  viewpoint.  The  influence  of 
advanced  weapons  and  computer  systems  on  the  mission  planning  process  was 
significant,  both  from  a  negative  and  positive  perspective.  On  the  negative  side,  the 
increased  number  of  advanced  weapons  being  used  by  the  Air  Force  greatly  increased  the 
infomiation  requirements  of  the  mission  planning  process.  For  instance, 

extensive  use  of  precision  munitions  created  a  requirement  for  much  more 
detailed  intelligence  than  had  ever  existed  before.  It  is  no  longer  enough  for 
intelligence  to  report  that  a  certain  complex  of  buildings  housed  parts  of  the 
Iraqi  nuclear  program;  targeteers  now  want  to  know  which  part  of  the  building, 
since  they  have  the  capability  to  strike  with  great  accuracy.  (Department  of 
Defense,  1992:xvi) 

The  increased  information  and  planning  detail  required  for  these  advanced  w  .capons  adds 
complexity  to  the  mission  planning  process.  This  complexity  lengthens  the  amount  of 
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time  a  planner  needs  to  produce  a  plan.  In  addition  to  increasing  planning  complexity 
and  planning  time,  the  need  for  more  information  increases  the  need  for  communication 
and  coordination.  As  a  result,  communications  capabilities  and  computer  networks  must 
expand  to  support  the  trend.  On  the  positive  side,  increases  in  computer  technology  and 
improvements  in  communication  help  reduce  the  planners’  workloads  and  allow  them  to 
manage  the  additional  task  complexity. 

Desert  Storm  showed  that  the  effectiveness  of  our  forces  is  increased  by  these 
high-technology  systems  (E)epartment  of  Defense,  ]992:xii).  Interestingly,  many  specific 
mission  planning  lessons  discussed  in  the  other  five  categories  can  be  directly  related  to 
the  increased  use  of  these  high-technology  systems.  Many  of  the  mission  planning 
problems  later  discussed  are  the  result  of  tiying  to  control  either  the  increased 
information  needs  of  new  systems  or  the  large  amounts  of  data  that  are  generated  by  new 
systems. 

Automation.  Computers  and  softwaie  for  automated  flight  planning  began  to 
significantly  change  unit-level  planning  in  the  1980s.  These  systems  allow  the 
traditionally  manual  tasks  of  route  planning,  weapons  planning,  threat  analysis,  and 
preparation  of  crew  materials  to  be  performed  with  the  aid  of  a  computer.  The  results  are 
transferred  into  printed  and/or  digital  form  for  use  by  the  aircrews  and  aircraft  after 
planning  is  complete. 

Prior  to  Jesert  Storm,  system  developers  assumed  that  automated  planning 
systems  would  significantly  reduce  the  planner's  workload  and  allow  more  time  to 
develop  and  brief  tactics.  Unfortunately,  this  reduction  could  not  be  quantified  during 
Desert  Storm  for  two  reasons.  First,  conducting  a  time  study  or  data  collection  effort 
duting  combat  operations  was  not  practical  nor  attempted.  Second,  because  mission 
planning  times  vary  widely  depending  upon  aircraft  type,  mission  type,  weapon  type,  and 
other  factors,  results  from  any  one  study  would  have  been  difficult  to  generalize  to  other 
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siuialions.  However,  the  subjective  opinions  from  Desert  Storm  Mission  Support  System 
(MSS)  users  suggest  automated  planning  systems  did  significantly  reduce  the  planner’s 
workload  and  increase  tactics  development  time  (Fermenter,  1991:12). 

During  Desert  Storm,  at  least  four  mission  planning  systems  were  used  by  US  Air 
Force  fighter  aircrews.  FPlan  (PC  Based),  Mission  Support  System  (MSS)  I  (Cromemco 
CS  2S0  based),  and  the  MSS  II  (Cromemco/DEC  MicroVax  II  based)  were  widely  used 
by  unit-level  mission  planners  at  TAF  fighter  units  (Fermenter,  1991:2).  In  addition,  a 
system  developed  exclusively  for  the  F-1 17  was  used  by  F-1 17  crews.  While  little 
information  was  available  on  the  F-1 17  mission  planning  system's  performance,  the  DOD 
report  to  Congress  stated  that  The  mission  planning  system  designed  specifically  for  the 
F-1 17  is  time  consuming  and  labor  intensive”  (Department  of  Defense,  1992:T-74). 

Of  the  three  more  widely  used  systems,  the  MSS  II  provides  the  most  advanced 
capability.  In  his  report  on  Desert  Storm  Mission  Planning  Lessons  Learned,  Fermenter 
reports  that; 

The  MSS  was  an  invaluable  aid  to  mission  planning  for  a  variety  of  flying 
units  participating  in  DS  [Desert  Storm].  In  most  cases,  the  MSS  saved  time 
and  minimized  errors  that  otherwise  would  have  occurred  in  the  planning 
process.  (Permenier,  1991:6) 

As  further  support  to  the  concept  of  increased  automation  of  the  mission  planning 
process.  Brig  Gen  Lee  A.  Downer,  Joint  Task  Force  (JTF)  Proven  Force  commander, 
stated  in  his  Air  Power  Journal  article: 

The  money  invested  in  advanced  intelligence  and  mission-planning  systems  is 
proving  to  be  well  spent.  The  growth  potential  in  the  systems  for  intelligence 
data  management,  threat  and  target  analysis,  mission  planning,  and  campaign 
planning  will  keep  operational  intelligence  people  busy  for  some  time  to  come. 

We  must  continue  to  stay  on  the  edge  of  this  exciting  technology.  ( 1991 : 12) 

Despite  the  general  success  of  these  automated  planning  systems,  there  were  some 
repotted  problems  with  their  use  and  shortfalls  in  their  capabilities.  Some  of  these 
problems  were  support  issues  such  as  inadequate  operator  and  system  manager  training. 
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Others  were  more  specific,  such  as  an  inadequate  threat  update  capability,  lack  of 
accurate  threat  models,  and  the  absence  of  a  multiple  route  coordination  capability. 

Some  of  these  problems  are  mentioned  later  in  specific  sections  of  this  discussion. 
Fermenter  provides  a  more  comprehensive  discussion  of  specific  system  problems  and 
shortfalls. 

Communication.  A  great  deal  of  the  mission  planning  communication  (both 
secure  and  non-secure)  between  units,  or  with  higher  headquarters,  retied  on  the  use  of 
telephone  lines.  Subsequently,  public  and  military  telephone  lines  were  almost  in 
constant  use  and  in  short  supply  (Fermenter,  1991:33).  This  shortfall  was  especially  true 
for  transmitting  large  files,  databases  (e.g.,  ATO,  threats)  or  imagery  which  required 
circuits  with  wide  bandwidth  for  efficient  transmission  (Department  of  Defense, 

1992: 140).  This  issue  is  equally  valid  when  discussing  voice  or  data  communication  and 
is  substantiated  by  reports  outside  the  mission  planning  realm.  As  addressed  in  later 
discussions  about  mission  planning  information  flow,  the  lack  of  telephone  circuits 
became  a  mtyor  impediment  to  providing  planners  with  accurate  and  timely  information. 

Mission  Planning  Information  Flow.  There  were  substantial  problems  with 
the  timeliness  and  accuracy  of  mission  planning  information  during  Desert  Storm. 
Problems  with  information  flow  were  not  limited  to  mission  planning.  It  was  a  theater- 
wide  problem.  As  mentioned  in  DOD's  report  to  Congress, 

At  the  beginning  of  Operation  Desert  Shield  force  deployment,  there 
essentially  was  no  existing  US  military  command,  control,  communications, 
and  computer  (C4)  infrastructure  in  the  region.  By  mid-January,  the  Coalition 
had  established  the  largest  tactical  C4  network  ever  assembled. . .  Despite  this 
effort,  the  start  of  Operation  Desert  Storm  made  it  clear  the  requirement  fur 
communications  outstripped  the  capacity.  This  was  especially  true  for  the 
large  amounts  of  imagery  and  intelligence  data  bases  that  needed  to  be 
transmitted  throughout  the  theater.  (Department  of  Defense,  1992: 140) 

Air  Tasking  Order.  The  ability  to  task  nil  flying  units  (Army,  Navy 
and  Air  Force)  in  a  theater  of  operations  as  large  as  Desert  Storm's  with  a  single 
comprehensive  and  coordinated  docume^/  wis  effective  and  considered  successful 
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(Department  of  Defense,  1992:138).  Nevertheless,  there  were  many  problems.  The 
ATO's  size  and  complexity  caused  delays  in  building  the  document  a.s  well  as 
disseminating  it  to  the  lower-level  planners. 

Problems  were  encountered  before  the  ATO  was  received  at  the  wing  level.  The 
equipment  at  the  force-level  was  not  sufficient  for  creating  a  document  .tf  this  size.  In 
addition,  the  complexity  of  coordinating  all  forces  caused  delays  in  issuing  the  ATO  on 
time.  While  the  ATO  was  usually  transmitted  electronically  to  units  every  day  from  1700 
to  1900  hours,  some  units  (B-S2  units  and  Navy  units  on  the  carriers)  could  not  receive 
the  document  electronically  due  to  equipment  interoperability  problems  (Department  of 
Defense,  1992:139).  Special  couriers  were  sometimes  employed,  but  couriers  added 
several  hours  to  the  dissemination  time.  L^ter  in  the  conllict  this  problem  was  overcome 
by  extending  CENTCOM's  tactical  super-high  frequency  satellite  communications 
(SATCOM)  network.  Even  when  electronic  means  were  available,  transmission  was 
slower  than  desired  due  to  the  large  size  of  the  files  and  inadequate  communication  line 
capacity . 

Lack  of  timely  Battle  Damage  Assessments  (BDA)  at  the  force-level  planning 
level  lessened  the  content  and  detail  contained  in  the  ATO.  These  shortfalls  had  a 
secondary  effect  on  the  w'ing  and  squadron  mission  planners.  Target  selection  and 
planning  at  force-level  were  almost  complete  before  the  results  of  previous  missions  were 
available  to  the  force-level  planners.  Therefore,  force-level  planners  needed  to  build  in 
execution  flexibility  and  responsiveness  into  the  ATO.  “Planners  built  flexibility  and 
responsiveness  into  operations  by  delegating  most  detailed  mission  planning  to  the  wing 
and  unit  level”  (Department  of  Defense,  1992:140).  While  this  delegation  of 
responsibility  allowed  the  force-level  planners  to  meet  daily  ATO  release  times,  it 
increased  the  workload  and  data  requirements  of  planners  at  the  unit-level. 

Once  the  wing  finally  received  the  ATO,  an  inordinate  amount  of  time  was  spent 
breaking  down  the  document  and  understanding  the  information  (Pemienter,  1991:v). 
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This  inordinate  amount  of  time  v*'as  caused  by  two  factors.  First,  because  many  units 
were  involved  in  the  operation,  the  document  was  large-anywhere  between  300  to  600 
pages  in  length.  Second,  the  mostly  manual  methods  of  shredding  out  the  ATO  and 
processing  it  locally  were  slow.  Desert  Storm  participants  concluded  that 

the  mission  planning  process  would  be  more  efficient  if  ATO  receipt  was  more 
timely  and  local  automated  processing  aids  were  available.  Lack  of 
communications  support  to  disseminate  the  ATO  and  lack  of  automation 
support  to  process  it  locally  per  unit's  needs  limited  the  time  available  for 
initial  mission  planning  and  subsequent  updates.  (Fermenter,  1991  :v) 

Intelligence  Data.  At  the  DOD  level,  the  intelligence  community  was 
ill-prepared  to  handle  the  large  volume  of  intelligence  information  that  was  needed  by 
commanders  and  planners  during  Desert  Storm  (Department  of  Defense,  1992:C-4,  C-10, 
C-18).  With  respect  to  unit-level  mission  planning  in  particular,  this  inadequate 
preparation  was  particularly  evident.  Despite  the  fact  that  our  tactical  intelligence 
systems  have  never  been  more  capable,  problems  with  intelligence  prioritization, 
dissemination,  timeliness,  accuracy,  and  system  interoperability  plagued  unit-level 
mission  planners  (Department  of  Defense,  1992;C-10;  Fermenter,  1991:v,  9). 

Threat  Location  Data.  Aircrews  Involved  in  mission  planning 
at  the  unit  level  frequently  mistmsted  the  threat  location  data  they  received.  This  mistmst 
was  due  to  the  perception  that  the  data  was  not  current  or  consistent.  Aircrews  reporting 
back  from  personal  accounts  often  reported  the  published  threat  locations  were  inaccurate 
or  had  not  been  updated  (Fermenter,  1991:29). 

Few  of  the  units  that  had  automated  mission  planning  systems  with  threat  display 
and  route  optimization  capability  actually  used  it.  This  reluctance  to  use  these  decision 
aids  was  primarily  due  in  part  to  the  mistrust  of  the  threat  data.  Additionally,  a  lack  of 
system  training,  interoperability  problems  between  the  mission  planning  systems  and 
unit-level  intelligence  computers,  poor  threat  models,  and  a  time  consuming  and  difficult 
database  administration  task  reduced  the  utility  of  this  capability  (Fermenter,  1991  :v,  vii. 
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19-20, 29-3 1 , 34-37).  In  the  end,  many  unit-level  planners  fell  back  on  techniques  that 
have  served  them  well  for  years-band-drawn  wall  charts  and  close  interaction  with  their 
intelligence  support  personnel. 

Mapping.  Charting  and  Geodesy  fMC&GV  Maps  and 
charts  have  always  been  important  lo  mission  planners.  Desert  Storm  reaffirmed  that 
fact.  Planning  for  precision  guided  munitions  and  night-time  navigation  requires  accurate 
point  positioning  and  detailed  MC&G  products.  These  detailed  products  were  sometimes 
not  available  when  needed.  Shortages  of  some  paper  maps  and  charts  occurred  in  the 
detailed  scales  (1:50,000).  These  shortages  were  overcome  at  times  by  the  use  of  wide- 
coverage  satellite  photography  which  had  been  acquired  from  U.S.,  foreign,  and 
commercial  sources  and  processed  to  meet  accuracy  requirements  (Department  of 
Defense,  1992:C-5;  Fermenter,  1991:29). 

On  a  positive  note,  the  use  of  digital  mapping  products  to  replace  paper  maps  and 
charts  during  flight  planning  was  successful.  As  with  the  paper  charts,  problems  creating, 
distributing,  and  updating  these  maps  also  existed.  However,  digital  mapping  products 
provided  additional  benefits  over  their  paper  counterparts.  These  digital  maps  and  charts 
can  be  used  in  the  automated  mission  planning  computers  and  printed  with  the  route  or 
threats  overlaid  on  them  for  use  during  the  mission.  Digital  map  manipulation 
capabilities  greatly  reduced  the  workload  on  the  flight  planners  who  previously  had  to 
draw  the  routes  and  threats  by  hand. 

Imagery.  The  need  for  wide-area,  high  resolution  imagery  in 
addition  to  target  area  photos  was  validated  during  Desert  Storm.  Mission  planners  have 
long  demanded  high  resolution  photos  of  specific  targets  to  perform  weapons  planning 
and  targeting.  However,  wide-area  imagery  of  sufficient  resolution  has  only  been 
recently  available  to  unit-level  mission  planners  for  wide  aiea  and  precision  control 
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navigation.  This  wide-area  imagery  was  found  helpful  for  area  orientation,  terrain 
perspective  viewing,  and  large  area  targets  (Fermenter,  1991:28). 

During  Desert  Storm,  unit-level  planners  had  access  to  several  sources  of  wide- 
area  photographs.  These  included  foreign,  commercial,  and  U.S.  military  sources .  Two 
of  the  more  common  sources  of  this  wide-area  imagery  outside  the  military  were 
LANDSAT  and  SPOT.  LANDSAT  is  a  U.S.  commercial  satellite  managed  by  the 
Department  of  Commerce,  National  Oceanographic  and  Atmospheric  Administration  that 
is  capable  of  providing  a  30  meter  resolution  scene  of  a  185km  by  170km  area.  SPOT 
{Satellite  Probatoire  dVbservatlon  de  la  Terre)  is  a  French  commercial  satellite  that  is 
capable  of  providing  a  10  meter  resolution  scene  of  approximately  60nm  by  60nm 
(Department  of  Defense,  1992:C-5). 

Neither  of  these  two  sources  fully  met  Air  Force  mission  planning  navigation 
requirements.  Nonetheless,  the  use  of  SPOT  imagery  aligned  with  special  control  point 
data  and  formatted  digitally  for  use  on  mission  planning  computers  was  an  ^'invaluable 
asset  to  the  offensive  air  campaign"  (Cosby,  1991:7).  Units  equipped  with  this  capability 
“used  the  database  exclusively  for  planning  POM  [precision  guided  munitions]  attacks, 
precision  navigation,  and  locating  high  value  targets.  This  capability  achieved  impressive 
results  when  combined  with  an  automated  planning  system”  (Fermenter,  1991:27-28). 

While  this  capability  demonstrated  the  effectiveness  of  imagery  for  mission 
planning  purposes,  its  success  was  tempered  by  the  difficulties  associated  with 
production  and  dissemination  of  the  imagery  to  unit-level  mission  planners  (Fermenter, 
1991:49).  Again,  imagery  dissemination  was  not  only  a  unit-level  mission  planning 
problem.  The  DOD  report  to  Congress  on  Desert  Storm  mentions  that  difficulties 
occurred  in  disseminating  national  and  theater  intelligence,  and  numerous  incompatible 
secondary  imagery  dissemination  systems  existed  within  the  theater  (Department  of 
Defense,  1992:C-18). 
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Production  of  I'.ie  184  scene  SPOT  imagery  database  for  the  mission  planning 
computers  was  excessively  time  consuming.  While  the  database  for  Iraq  was  available  to 
selected  Air  Force  units  at  the  beginning  of  Desert  Storm  ( 17  Jan  91),  its  production  had 
started  many  months  earlier.  Transmission  of  a  SPOT  image  from  one  unit  to  another,  or 
from  higher  headquarters  to  a  subordinate  unit,  typically  exceeded  30  minutes  per  scene 
over  a  KY-68  secure  tactical  telephone  system.  Even  with  the  use  of  software 
compression  techniques  (8:1  compression  ratio)  this  time  rarely  could  be  reduced  to  less 
than  15  minutes.  At  times,  transmission  was  not  possible  due  to  fluctuating  telephone 
circuit  quality  (Fermenter,  1991:49;  Cosby,  1991:5). 

In  addition  to  the  difficulties  in  acquiring  wide-area  imagery  for  mission  planning, 
similar  problems  existed  with  obtaining  timely  reconnaissance  photos  for  assessing  battle 
damage.  Inadequate  communication  to  the  tactical  level  hampered  the  targeting  and 
planning  task  that  had  been  delegated  from  the  theater  level  to  the  wing.  As  reported  by 
the  DOD  to  Congress, 

planners  at  the  air  wing  level  often  were  forced  to  rely  on  cockpit  video,  pilot 
reports,  and  limited  organic  intelligence  and  planning  capabilities  to  choose  the 
best  attack  options  and  aimpoints.  Doing  that  required  access  to  recent  target 
imagery  and  BDA  information,  which  often  were  neither  timely  nor  adequate. 

At  times,  this  led  to  unnecessary  restrikes.  (Department  of  Defense,  1992:C-21 ) 

Despite  the  collection,  processing  lead  time,  and  dissemination  problems  that 
occurred  during  Desert  Storm,  the  Air  Force  emerged  with  a  much  clearer  picture  of  its 
unit-level  mission  planning  imageiy  requirements.  Future  mission  planners  can  be 
expected  to  increasingly  demand  more  imagery  to  accomplish  their  tasks  (Fermenter, 
1991:viii). 

Weather  Information.  As  learned  in  previous  conflicts  throughout 
history,  the  effects  of  the  weather  on  mission  execution  should  be  considered  during 
tactical  planning.  Forecasts  based  on  historical  records  of  Iraqi  weather  patterns 
predicted  that  the  weather  should  not  have  been  a  major  factor  during  Desert  Storm. 
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However,  during  execution  of  the  operation,  Iraq  experienced  its  worst  weather  in  14 
years.  Over  15%  of  scheduled  aircraft  attack  sorties  were  cancelled  during  the  first  10 
days  because  of  low  overcast  skies  or  poor  visibility  (Department  of  Defense,  1992:227). 

Correctly  forecasting  weather  patterns  along  the  flight  path  and  in  the  target  area 
is  critical  to  the  success  of  any  mission.  Poor  weather  can  severely  limit  a  weapon 
system's  effectiveness,  require  a  mission  abort,  or  can  delay  aspects  of  a  mission  that  may 
be  time  critical.  Knowing  when  these  conditions  may  exist  and  when  to  consider  them 
constraints  during  mission  planning  is  becoming  more  important.  In  addition,  the  need 
for  specialized  weather  data  and  information  is  increasing  (Fermenter,  1991 : 15). 

This  increased  reliance  on  timely  weather  forecasts  and  weather  data  is  driven 
primarily  by  the  Air  Force's  increased  use  of  precision  guided  munitions.  Traditional 
weather  forecasts  are  consider  to  be  sufficient  for  traditional  route  planning.  However, 
the  information  required  to  conduct  munitions  planning  for  sophisticated  weapons 
requires  the  use  of  computerized  decision  aids  (Fermenter,  1991:15).  These  decision  aids 
in  turn  increase  the  need  for  specialized  data  forecasts  of  target  area  weather  parameters 
such  as  humidity  levels,  winds,  temperatures  (ground  surface  and  air  temperatures),  and 
other  weather  data  which  is  combined  with  intelligence  information  about  the  target  to 
estimate  detection  ranges,  lock-on  ranges  and  other  planning  details. 

Organization  and  Staffing.  Three  organizational  elements  significantly 
impact  the  effectiveness  of  unit-level  mission  planning;  mission  planning  cells,  mission 
commanders,  and  system  managers. 

Mission  Planning  Cell.  During  Desert  Storm,  many  units  adapted 
their  wing  or  squadron  organizations  to  include  the  use  of  a  Mission  Planning  Cell 
(MFC).  The  MFC  normally  consisted  of  several  crew  members  and  intelligence  support 
personnel  who  augment  either  the  mission  commander,  or  the  individual  flight  planners 
by  accomplishing  preliminary  flight  planning.  These  MFCs  normally  worked  on  the 
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plans  before  the  start  of  the  squadron  duty  day  and  passed  the  mission  packages  to  the 
flight  commanders  for  final  preparation.  Depending  upon  the  structure  of  the  wing, 

MFCs  were  used  at  both  the  wing  and  squadron  level;  however,  seldom  at  both. 

While  not  all  units  chose  to  use  the  MFC  concept,  those  units  that  did,  reported 
MFCs  were  successful.  Mission  Planning  Cells  offer  some  advantages  over  individual 
mission  planning  approaches:  their  ability  to  support  around-the-clock  operations,  more 
efficient  data  handling,  and  better  time  management  (Fermenter,  1991  ;v). 

Mission  Commander.  During  Desert  Storm,  the  Mission  Commander 
emerged  as  a  key  person  within  the  unit-level  mission  planning  process.  This  individual 
was  responsible  for  package  planning,  coordination  and  execution  of  the  mission  as 
tasked  in  the  ATO.  Usually  one  of  the  wing's  most  experienced  pilots,  the  Mission 
Commander  worked  closely  with  the  MFC  (if  used)  and  performed  most  of  the  external 
coordination  with  higher  headquarters  and  supporting  units.  Increased  communication 
and  better  automated  aids  were  mentioned  as  ways  to  enhance  his  capacity  to 
comprehend  the  overall  situation  and  to  improve  his  decision-making  ability  (Fermenter, 
1991:v). 

System  Managers.  System  managers  are  the  individuals  responsible 
for  maintaining  the  operational  databases  on  the  automated  mission  planning  systems. 
They  were  in  essence,  the  system  experts  and  were  expected  to  serve  as  troubleshooters, 
trainers,  and  scapegoats.  These  individuals  were  normally  assigned  this  Job  in  addition  to 
tlusir  nomial  flying  or  other  full  time  duties  and  had  little,  if  anyone,  to  back  them  up.  In 
addition,  the  job  typically  demanded  more  time  than  was  available  outside  their  flying 
duties.  Reports  from  individuals  working  with  the  MSS  show  that  unit  commanders  did 
not  recognize  the  system  manager's  importance  until  “after  units  began  to  use  the  MSS 
and  other  systems  seriously.” 

On  the  basis  of  on  the  information  gathered  from  the  MSS  system  managers  after 
Desert  Storm,  some  qualifications  are  emerging  for  system  managers.  They  should  be 
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competent  in  both  the  software  and  hardware  aspects  of  the  system:  knowledgeable  about 
flight  operations  (although  not  necessarily  rated);  hold  a  full  time  position;  and  be 
iamiliar  with  intelligence  information  (maps,  targets,  imagery,  threats,  etc.)  (Fermenter, 
1991:11). 

Training  and  Proficiency.  We  did  not  find  evidence  that  aircrews  lacked 
proficiency  in  manually  performing  basic  mission  planning  functions.  There  were, 
however,  some  problems  performing  the  mission  planning  functions  on  the  various 
automated  mission  planning  systems.  Most  of  these  problems  were  attributable  to  either 
system  problems  (hardware  or  software)  or  operator  inexperience  and  lack  of  training  on 
the  systems. 

According  to  many  users,  both  system  manager  and  operator  training  for  the  .MSS 
was  inadequate.  Units  never  established  formal  training  programs  before  Desert  Storm. 
The  only  formal  training  program  was  initial  training  for  system  managers  provided  by 
HQ  TAC.  Subsequently,  most  training  that  users  obtained  on  the  automated  mission 
planning  systems  occurred  on  the  job,  and  was  conducted  by  the  system  managers  as  time 
allowed. 

User  and  system  manager  proficiency  on  the  MSS  automated  mission  planning 
system  “varied  extensively  a.mong  the  units,  particularly  with  regard  to  the  abilities  of  the 
system  managers”  (Fermenter,  1991:41).  Interestingly,  Fermenter  stated  there  was  no 
correlation  between  the  proficiency  of  system  managers  and  how  they  felt  about  training 
they  had  received. 

On  the  basis  of  on  these  observations.  Fermenter  suggests  that  the  need  for  a 
formal  training  program  be  reevaluated,  that  increased  emphasis  be  placed  on  unit 
training  within  the  unit,  that  automated  mission  planning  training  be  included  in  Mission 
Qualification  Training  (MQT),  and  that  the  training  programs  be  evaluated  during  annual 
readiness  and  administrative  exercises/inspections  (Fermenter,  1991:42). 
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Analysis  and  Conclusions  from  Operational  Review.  We  discovered 
several  trends  when  we  analyzed  mission  planning  lessons  from  Desert  Storm  and  the 
current  operational  process.  These  trends  may  be,  as  Secretary  of  Defense  Cheney 
proposed,  an  indication  of  what  military  capability  is  needed  in  10, 20,  or  30  years.  We 
suggest  that  each  of  these  trends,  and  their  impact  on  alternatives,  should  be  considered 
when  comparing  solutions  to  various  mission  planning  problems  or  when  evaluating  a 
current  process  against  mission  requirements. 

Mission  Planning  Trend  #1.  The  number  of  decisions  to  be  made  by 
mission  planners  at  the  unit  level  is  increasing.  As  the  tempo  of  planning  increases  to 
match  the  tempo  of  war,  and  more  demand  is  placed  on  our  command  and  control 
mechanisms,  more  decisions  are  being  delegated  to  lower  levels.  This  delegation  is 
driven  by  the  requirement  to  use  the  most  timely  and  accurate  information  available  for 
planning  decisions  and  the  continued  difficulty  in  obtaining  BDA  at  the  upper  echelons  of 
command.  It  is  also  an  attempt  to  ensure  that  executable  force*level  plans  are  developed 
from  inputs  received  from  the  unit  level.  This  solicitation  for  inputs  from  the  unit  level  is 
confirmed  by  Maj  Gen  Glossen,  who  after  describing  the  overall  Desert  Storm  air 
campaign,  gave  the  following  instructions  to  wing  commanders;  “This  is  your  plan,  and 
you  have  the  option  of  asking  me  to  change  anything  you  want  to.  Nothing  is  sacrosanct. 
We're  just  trying  to  do  this  the  smartest  way  possible”  (Morrocco,  1992:73). 

This  shift  in  decision  authority  is  also  consistent  with  the  Air  Force's  new  concept 
of  the  Composite  Wing.  As  ’ed  by  Air  Force  Chief  of  Staff  General  McPeak, 

Much  may  be  said  about  the  operational  advantages  of  such  a  wing,  but  of 
primary  importance  is  the  prospect  it  presents  of  being  able  to  reform  the 
command  and  control  system  by  cutting  back  sharply  on  the  need  for  detailed 
guidance  from  above,  (McPeak,  1990:8) 

Mission  Planning  Trend  #2.  The  amount  and  level  of  detail  of 
mission  planning  information  required  by  mission  planners  at  the  unit  level  is 
increasing.  This  increase  is  attributable  to  two  causes.  One  is  the  increase  in  the  number 
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of  decisions  mentioned  in  Mission  Planning  trend  #1 .  As  the  number  of  decisions 
increases,  so  too  does  the  need  for  information  to  make  those  decisions.  The  second 
factor  behind  the  need  for  more  detailed  information  is  characteristic  of  the  more 
advanced  weapons  we  seek  to  use.  The  more  accurate  and  advanced  the  weapon  system, 
the  more  accurate  and  timely  the  information  that  it  uses  must  be.  As  advanced  weapons 
such  as  PGMs  increase  in  use,  the  information  requirements  of  the  system  planners  is  also 
increasing. 

Mission  Planning  Trend  #3.  The  use  of  Decision  Support  Systems 
(DSS)  by  mission  planners  at  the  uniUlevel  is  increasing.  As  the  number  of  decisions 
planners  make  increases,  and  the  amount  of  information  used  by  the  planners  increases, 
the  workload  of  mission  planners  also  increases.  Decision-makers  are  turning  to 
technology  to  help  manage  the  information  and  present  the  complexities  in  a  way  that 
assures  timely  decisions  are  made.  Examples  of  this  trend  are  already  evident.  DSS,  like 
mission  planning  computers,  Electro-Optical  Tactical  Decision  Aids  (EOTDAs),  and 
intelligence  analysis  systems,  are  already  used  at  the  unit-level  to  manage  information 
and  aid  planners  and  analysts. 

Mission  Planning  Trend  #4.  The  requirement  for  more 
communication  capabilities  and  interoperability  tol from! within  the  unit-level  is 
increasing.  As  the  tempo  of  planning  activities  increases  and  the  amount  of  information 
needed  by  unit-level  planners  increases,  the  need  to  share  information  between  planners 
also  increases.  Also,  while  the  amount  of  direction  flowing  from  higher  headquarters 
down  to  the  unit-level  is  expected  to  decrease,  no  reduction  in  the  amount  of  reporting  or 
feedbac  k  from  the  unit-level  to  higher  headquarters  is  anticipated.  In  fact,  the  opposite  is 
likely.  The  net  effect  will  be  an  overall  increase  in  the  need  for  more  communication 
capabilities  and  interoperability  to  handle  the  additional  requirements. 

Mission  Planning  Trend  #5.  The  use  of  group  punning  approaches 
(e.g.,  a  Mission  Planning  Cell)  at  the  unit-level  is  increasing.  As  plan;ung  workload 
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increases,  mission  planners  divide  the  workload  among  a  number  of  individuals  lo  try  to 
reduce  the  complexities  and  handle  (he  increase  in  information.  By  dividing  the  planning 
functions  and  delegating  them  to  individuals,  planners  can  captitalize  on  tlte  idea  of 
increased  efTectiveness  through  specialization.  This  trend  can  be  seen  in  the  increased  use 
of  MFCs  during  Desert  Storm 

Mission  Plannintt  Trend  #6.  Behavioral  faaors  of  the  unit-level 
mission  planning  process  are  becoming  more  important  in  increasing  process 
effectiveness  (or plan  quality).  As  the  mission  planning  process  becomes  more  and  more 
effective  through  the  applied  use  of  advanced  technologies  (DSS,  Increased 
interoperability,  faster  computers,  more  information),  less  gain  in  process  effectiveness 
(or  plan  quality)  will  be  attributable  to  tliose  technologies.  Improvements  to  overall 
process  effectiveness  must  therefore  come  from  outside  those  sources.  Therefore,  with 
fewer  opportunties  to  improve  mechanical  factots  in  planning,  behavioral  factors  such  as 
fatigue,  group  interaction,  sender/receiver  biases,  or  learning  capacity  become 
increasingly  important  to  improving  process  effectiveness. 

Mission  Planning  Tren(j  #7.  Mission  planner  computer  proficiency 
and  training  are  becoming  more  important  in  determining  process  effectiveness  (or 
plan  quality).  As  identified  in  the  mission  planning  lessons  of  Desert  Storm,  improving 
planner  computer  proficiency  through  better  training  is  needed.  As  more  computer 
systems  or  decision  support  systems  are  introduced  into  the  mission  planning  process, 
attention  to  training  may  be  one  approach  for  easily  improving  performance.  Lessons 
learned  indicate  that  inexperienced  personnel  and  inadequate  automated  planning  system 
training  caused  problems. 

Revievw  of  Academic  Literature 

While  our  review  of  operational  literature  provides  the  foundation  for  Objective 
#  ! 's  conceptual  model  of  the  ACMP  process,  our  review  of  academic  literature  in  this 
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section  primarily  builds  the  groundwork  for  deriving  Objective  #2’s  evaluation 
methodology.  The  general  format  for  our  review  of  academic  literature  is  to  initially 
address  the  umbrella  concept  of  Command  and  Control  (C^)  and  then  narrow  to 
subsidiary  elements.  Through  this  process,  we  will  progress  through  the  following  steps: 
C^,  planning,  decision  theory,  and  decision-aiding.  Generally,  within  each  of  these  steps 
the  review  progresses  from  general  theory  to  application  issues  and  evaluation  methods. 
Of  course,  given  our  objective  to  derive  from  previous  research  a  general  method  for 
evaluating  process  effectiveness  of  an  ACMP  process,  evaluation  is  a  primary  focus  of 
our  review.  However,  the  following  summary  from  the  1978  Defense  Science  Board 
study  of  command  and  control  system  management  illustrates  the  difficulty  of  our  task: 

The  absence  of  commonly  understood  concepts  of  command  and  control 
system  performance  and  the  existence  of  language  barriers  among 
technologists,  policy  analysts,  planners  and  commanders  all  underlie  the  fact 
that  we  lack  in  DOD  any  very  useful  conceptual  framework  for  evaluating  or 
specifying  command  and  control  systems.  (Mull,  1982:23) 

Command  and  Control.  Within  ACC’s  new  nomenclature  that  no  longer 
draws  distinctions  between  tactical  and  strategic  components,  the  unit-level  mission 
planning  procc.ss  is  part  of  the  overall  Theater  Air  Control  System  (TACS)-the  Air 
Force's  theater  Command  and  Control  (C^)  system.  For  that  reason,  we  begin  our 
literature  review  discussion  with  an  overview  of  Theory.  Our  review  of  command 
and  control  concludes  with  a  summary  of  C^  evaI»:ation  approaches. 

Theory.  Hwang  et  al.  assert  that  computer  technology  has  been  applied 
to  command  and  control  “de.spite  the  absence  of  a  commonly  accepted  unifying  theory  or 
analytical  concept”  (1982:xv-\vii).  This  opinion  is  shared  by  Dr.  Stuart  Brodsky,  who 
observes  that  ‘designers  are  still  seeking  a  firm  theoretical  base  providing  clear  and 
precise  definitions  for  common  interpretation  and  understanding  of  ideas  and  quantitative 
methods  tor  investigating  relationships,  developing  insights,  and  establishing  design  and 
evaluation  piocedures.”  (Brodsky,  1982:42).  One  of  the  more  common  theoretical  bases 
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used  to  describe  or  discuss  C^,  is  that  of  Control  Systems  Theory.  Brodsky  explains  that 
**command  and  control  systems  have  three  basic  functions:  they  must  collect,  process  and 
correlate  data  and  information  to  provide  the  commander  with  a  picture  of  the  conflict 
environment  that  is  as  accurate  as  possible.”  Using  the  Control  Systems  approach,  he 
defines  five  basic  functional  elements  that  make  up  the  command  and  control  system:  1) 
the  environment;  2)  sensors;  3)  communications  and  data  bases;  4)  coiitroller/decision 
makers;  and  5)  control  inputs  (41-42). 

The  command  and  control  environment  changes  how  planning  systems  should  be 
developed  and  operated.  An  environment  of  substantial  uncertainty  differentiates 
command  and  control  planning  from  normal  managerial  planning.  Sutherland  argues  that 
robust  model  bases  are  critical  to  the  adaptive  planning  required  for  C^.  The  normally 
used  integral  model  base  structure  requires  development  of  detailed  scenarios  that  are 
matched  with  detailed  scripted  responses.  Sutherland  suggests  a  more  flexible  two  level 
model  structure  is  appropriate  for  C^  planning  systems.  The  first  level  has  categorical 
models  that  use  generic  scripts  to  quickly  respond  to  generic  referents.  The  user  can  then 
apply  template  driven  models  to  refine  the  generic  script  to  entirely  unanticipated  events 
(Sutherland,  1990:18-23). 

Evaluation  Approaches.  Several  evaluation  approaches  have  been 
proposed  for  evaluation.  Most  follow  the  General  Model  for  Evaluation  discussed  by 
Sprague  and  Carlson.  This  model  for  planned  experiments  is  widely  used  across  many 
fields.  Although  other  researchers  may  present  the  general  model  differently,  all 
generally  include  the  following  six  elements: 
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1 )  a  statsmeiU  of  objectives, 

2)  a  choice  of  measures, 

3)  a  choice  of  treatments  and  experinientai  units, 

4)  a  plan  for  assigning  treatments  to  experimental  units, 

5)  a  plan  for  selecting  the  experimental  units,  and 

6)  a  choice  of  analysis  criteria  and  techniques  (Spragtte  and  Carlson,  1982:  167). 

Some  researchers  focus  on  providing  overall  top-level  evaluation,  while  others 
address  specific  issues.  Researchers  have  found  development  of  evaluation  approaches 
that  balance  overall  evaluation  with  issue  analysis  to  be  difficult.  For  example,  in  the 
early  1970s  the  U.S.  Navy’s  Operational  Decision  Aids  project  initiated  a  contracted 
special  effort  to  ‘^develop  and  analyze  alternative  methods  for  measuring  the  performance 
of  existing  and  proposed  decision  aids”  The  Navy  abandoned  the  first  effort  which 
’’sought  to  foimulate  a  completely  comprehensive  basis  for  measure  of  effectiveness” 
because  it  was  ’’too  general.”  The  contractor  then  developed  a  methodology  which 
measured  the  ratio  of  reievant-to-total-information  retrieved.  However,  application  of 
this  measure  was  eventually  judged  impractical  and  its  use  was  suspended  (Sinaiko, 

1977:  5-6). 

D.  M.  Schutzer  specifically  reviewed  theory  in  attempting  to  define  top  level 
effectiveness  measures  for  C^.  Schutzer  states  command  and  control  enhances  battle 
performance  in  three  ways:  1 )  Enhanced  probability  of  survival,  2)  Enhanced 
effectiveness  (exchange  ratios),  and  3)  Improved  force  allocations  (percentage  of  assets 
that  can  be  assigned  to  given  engagement).  He  relates  these  enhancements  to  an  overall 
MOE  (ratio  of  relative  force  strength  before  and  after  engagement).  He  derives  a  force 
multiplier  effect  by  equating  to  equivalent  increase  in  units  involved  in  engagement. 
Schutzer’s  measure  empahsizes  that  our  systems  should  react  quickly  enough  to  change 
so  that  unopposed  evolution  of  the  situation  will  not  preempt  the  commander's  ability  to 
Implement  his  desired  first  response  (Schutzer,  1982: 1 18-144). 
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M.  L.  Metersky,  ia  a  thesis  entitled  A  Process  and  an  Approach  to  Design  and 
Evaluation,  adds  a  further  dimension  to  the  Control  Systems  approach  by  arguing  that  a 
system  is  composed  of  two  interdependent  parts:  1)  a  process  consisting  of 
people,  procedures,  and  organizations  and  2)  physical  components,  e.g.  sensors, 
computers,  and  communications.”  This  change  shifts  the  emphasis  from  the  physical 
attributes  of  the  system  to  human  factors  and  the  decision  maker.  Furthermore,  he  adds 
that  the  prime  consideration  of  a  process  is  to  provide  ”the  DM  [Decision  Maker]  with 
high  quality,  timely,  structured,  complete,  and  decision  specific  information,  within  the 
limits  of  his  cognitive  processing  capability.”  With  respect  to  theory  and  models, 
Metersky  says  that  “Control  systems  can  and  will  be  used  to  study  systems;  however, 
it  may  not  be  sufficiently  comprehensive  to  model  the  interactions  between  the 
proces.s  and  physical  components.  A  technique  that  combines  both  behaviotal  and 
control  systems  considerations  may  have  to  be  developed”  (Metersky,  1986:880-882). 

Measurement  of  may  also  require  a  second  look.  Metersky  contends  that  it  is 
inappropriate  to  measure  a  process  like  that  of  a  weapon  system.  “What  differentiates 
a  process  from  a  weapon  system  is  that  the  former  is  basically  people  intensive 
whereas  the  latter  is  hardware  intensive.”  “It  may  be  best  to  develop  a  procedure  that 
permits  a  process  measure,  a  system  measure,  as  well  as  a  weapon  system  measure 
to  develop  independently.”  The  independent  measures  of  {lerformance  (MOPs)  could 
then  be  merged  with  mission  performance  standards  to  produce  a  measure  of 
effectiveness  that  is  outside  the  boundary  of  the  system  because  “its  functions  are 
performed  in  the  context  of  an  operational  environment."  He  proposes  a  basic  measure  of 
process  effectiveness.  Decision  Quality.  Decision  Quality  is  composed  of  two 
vectors,  generic  and  mission.  The  generic  vector  has  behavioral  characteristics  that 
should  be  measured  with  social  science  methods  (e.g.,  questionnaires,  interviews).  The 
mission  vector  has  parameters  unique  to  one  or  more  missions  (e.g., tactics,  enemy  order 
of  battle).  He  suggests  the  factors  be  measured  using  a  Monte  Carlo  simulation,  man-in- 
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the-loop  war  gaming,  or  of>erational  exercises  (Metersky,  1986:881).  Metersky 
concludes  that,  as  a  minimum,  any  process  evaluation  methodology  should  possess 
the  following  characteristics; 

1)  provide  a  meaningful  measure  of  decision  quality; 

2)  allow  different  process  designs  to  be  compared; 

3)  be  acceptable  (understandable)  by  both  developer  and  sponsor; 

4)  include  consideration  of  (at  least)  mqjor  contributing  factors,  both  behavioral 
and  physical;  and 

5)  be  usable  across  missions  (Metersky,  1986:887). 

The  concept  of  separating  behavioral  and  mission  factors  has  been  used  by  other 
researchers  in  addition  to  Metersky  although  they  use  different  terms  to  describe  the 
vectors.  Adelman  et  aL  suggested  three  types  of  questions.  Their  questions  about  ease  of 
use  and  acceptability  to  the  decision  maker  are  behavioral,  while  performance  questions 
are  analogous  to  the  mission  vector  (Adelman  et  aL,  1961:61).  Adelman  and  Ulvila 
discuss  using  two  groups  of  criteria  for  evaluation  of  expert  systems:  performance  and 
usability  (Adelman  and  Ulvila,  1991:546). 

Planning.  As  previously  mentioned,  planning  is  a  subsidiary  part  of  which 
shifts  decisions  to  periods  of  reduced  activity  (Weissing  >Baylon.  1986:40).  As  a 
central  topic  of  our  thesis,  our  review  of  planning  literature  considered  both  planning 
theory  and  application  of  planning  in  the  military. 

Theoi'V.  After  many  years  of  writing  about  and  developing  the 
profession  of  planning  (primarily  urban  planning),  Melville  Branch  wrote  the  text  on 
general  theory  and  principles  of  planning  to  stimulate  the  study  of  comprehensive 
planning.  In  Comprehensive  Planning:  General  Theory  and  Principles,  he  states: 

“Little  has  been  produced  concerning  informational  and  analytical  procedures  and 
methods  commonly  employed  in  preparing,  maintaining,  and  revising  plans  whether  for 
business,  government,  or  the  military”  (1983:30).  Branch  provides  a  definitional  baseline 
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for  the  study  of  planning.  Many  make  establishing  objectives  the  first  step  of  planning. 
However,  within  Branch's  scheme,  objectives  are  dependent  variables  that  are  “dependent 
on  the  other  elements  and  considerations  in  the  analytical  loop,  and  therefore  subject  to 
frequent  change.  They  cannot  be  selected  as  a  matter  of  presumptive  preference  without 
reference  to  the  reality  of  their  effectuation'*  (76-77).  Branch  identifies  the  key  functions 
of  planning:  'The  most  essential  function  of  comprehensive  planning  is  the  allocation  of 
available  resources  among  competing  needs*'  (47).  He  also  identifies  important  issues. 
For  example,  he  shows  why  timing  is  a  critical  element  in  planning.  One  of  the  greatest 
threats  to  effective  planning  is  delay  that  results  while  attempting  to  develop  an 
analytically  complete  solution.  Branch  shows  that  all  elements  and  aspects  of  a  plan  can 
never  be  completely  analyzed  (1 17).  One  of  the  other  issues  planners  face  is  the  basic 
and  inevitable  human  resistance  to  planning.  Branch  provides  several  reasons  for  this 
resistance,  but  summarizes  that  generally  people  want  to  avoid  thinking  about  unpleasant 
possibilities  and  don't  wish  to  forego  present  needs  for  uncertain  future  needs  (192*193) 

Thomas  Saaty,  creator  of  the  Analytic  Hierarchy  Process  (AHP),  and  K.  Kearns 
review  current  theories  of  planning  in  their  book  on  applying  AHP  to  planning.  They 
review  other  definitions  (e.g.,  “process  for  determining  appropriate  future  action  through 
a  sequence  of  choices")  before  establishing  their  own.  They  define  planning  as  “a 
thinking  and  social  process  of  aligning  what  is  deduced  to  be  the  likely  outcome  of  a 
situation,  given  current  actions,  policies,  and  environmental  forces,  with  what  is 
perceived  as  a  desirable  outcome  which  requires  new  actions  and  policies"  (Saaty  and 
Kearns,  1985:95).  They  identify  three  prominent  planning  philosophies:  formal, 
incremental,  and  systemic.  Formal  philosophies  have  the  same  base  as  rational,  or 
normative,  decision-making.  In  formal  philosophy,  problems  can  be  quantified  and  the 
optimal  alternative  selected.  Incremental  planning  philosophies  are  at  the  other  extreme, 
assuming  a  lack  of  consensus  on  defining  the  problem  and  seeking  only  to  select  plans 
with  marginal  change  that  are  acceptable.  The  systemic  approach  endorsed  by  Saaty  and 
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Kearns  also  lacks  consensus  in  defining  problems  but  has  explicit  methods  for  structuring 
options.  Qualitative  and  quantitative  methods  are  used  to  identify  and  weight  criteria  that 
become  the  basis  for  plan  selection.  Systemic  methods  include  the  Delphi  method,  the 
Strategic  Assumption  Surfacing  and  Testing  (SAST)  method,  the  Symmetrical  Linkage 
System,  and  Multiattribute  Utility  Theory  (MAU).  Selection  of  an  appropriate  method 
depends  on  the  planning  situation.  Saaty  and  Kcams  identify  the  dichotomy  between 
“fact”  based  planning  and  “value”  based  planning  as  the  key  issue  that  separates  planning 
theory.  They  say  the  distinction  betwe<;n  facts  and  values  is  a  fiction  that  constrains 
planners  from  resolving  problems  which  are  inherently  conflictual.  Saaty  and  Kearns 
suggest  Saaty’s  Analytic  Hierarchy  Process  (which  is  predominantly  a  systemic  method) 
incorporates  the  best  of  the  three  planning  philosophies.  Us  strength  is  that  it  incorporates 
both  “facts”  and  “values"  into  the  planning  process  (Saaty  and  Kearns,  1985:99-127). 

Application  To  The  Military.  Stephen  Andriole,  tt  aL,  developed  a 
tactical  military  planning  process  model  in  conjunction  with  developing  tactical  expert 
system  aids  for  the  U.S.  Army.  The  researchers  video-taped  Army  War  College  exercises 
involving  six  participants  (3  instructors  and  3  students)  tasked  to  develop  a  tactical  plan 
for  defense  of  western  Europe.  Their  results  pointed  to  differences  between  military  and 
non-military  operational  planning.  They  found  military  planning  to  be  highly  stnicturcd. 
highly  repetitive,  mission-oriented,  and  verbal/graphic  (non-numeric).  They  contrast 
military  planning  attributes  to  non-military  planning  which  they  characterize  as  usually 
not  being  adversarial,  usually  not  having  high  accountability,  as  being  frequently  opinion 
driven,  and  as  usually  being  localized  with  one  or  two  planners.  While  the  researchers' 
results  are  preliminary,  the  results  support  the  idea  that  advanced  decision-aiding 
methodologies  and  microcomputing  can  be  combined  to  support  tactical  decision-making 
(Andriole  et  a/.,  1986:854-864). 
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Decision  Theory.  As  discussed  in  Chapter  I,  while  decision-making  is  only 
one  of  five  basic  steps  in  planning,  it  is  perhaps  the  most  important  element  for 
measuring  the  effectiveness  of  planning.  Our  conclusions  from  the  operational  review 
noted  the  trend  toward  increasing  the  number  of  decisions  made  at  the  unit  level  and  the 
increasing  complexity  of  those  decisions.  We  show  in  Chapter  IV*s  conceptual  model 
that  these  decisions  range  from  deciding  takeoff  weight  to  selection  of  sensor  data  for 
“smart”  munitions.  Given  the  Importance  of  decision-making  in  the  ACMP  process  and 
that  planning  shifts  decision-making  to  periods  of  reduced  activity,  an  understanding  of 
decision-making  is  required  to  be  able  to  evaluate  the  effectiveness  of  planning. 

However,  the  lack  of  academic  consensus  on  a  general  model  of  decision-making 
required  we  investigate  several  different  models. 

Theory.  Rational  models  of  decision-making  are  often  contrasted  with 
unstructured  (descriptive)  models.  Rational  models  describe  the  behavior  of  an  idealized 
person  who  expresses  consistent  preferences  and  consistent  beliefs  with  preferences  and 
beliefs  remaining  independent  from  each  other.  This  model  is  operationalized  into  the 
concept  that  people  are  able  to  evaluate  the  utility  of  each  decision  option  and  accurately 
evaluate  the  probability  of  each  option.  Given  these  premises,  the  rational  model 
suggests  individuals  will  make  “rational”  choices  seeking  higher  utility  (Hogarth, 
1983:65-67).  It  is  frequently  argued  that  rational  models  are  seldom  applied  in  reality.  In 
fact  it  is  often  said  that  people  are  generally  unaware  of  both  how  they  make  decisions 
and  why  they  prefer  one  alternative  over  another  (Hogarth,  1983:ix).  These  behavioral 
issues  will  be  further  considered  after  discussing  the  development  of  decision  theory. 

Modem  decision  theory  first  emerged  as  a  formal  discipline  in  the  middle  of  the 
twentieth  centuiy.  These  early  theories  were  solidly  based  on  the  axiomatic  structure  of 
rational  behavior.  In  the  sixties  the  theories  were  applied  to  well-structured  statistical  and 
managerial  problems.  Additionally,  researchers  began  to  examine  the  behavioral 
implications  of  the  theory,  particularly  the  areas  of  measuring  beliefs  and  tastes  which  are 


2-29 


integral  to  decision  theory.  In  the  seventies  and  eighties,  researchers  began  to  consider 
more  complicated  issues  such  as  multiple  decision-making  criteria  and  conflicting 
objectives.  These  additional  complexities  brought  the  need  for  increased  interdisciplinary 
cooperation  between  behavioral  scientists,  operational  researchers,  computer  scientists 
and  others  (Moskowitz  and  Bunn,  1987:249-258). 

Peter  Keen  discusses  the  the  rational  or  normative  models  that  are  the  theoretical 
foundations  of  optimization.  Keen  challenges  some  of  the  assumptions  that  Moskowitz 
and  Bunn  retain  in  suggesting  changes  which  are  more  radical  in  their  departure  from 
traditional  rational  theories.  However,  Keen  addresses  the  same  difficulties  of  multi¬ 
criteria  decision  complexity.  Keen  states  descriptive  models  such  as  Simon's 
“satisficing”  are  generally  accepted  as  far  more  accurate  descriptors  of  decision-making 
behavior.  He  believes  operations  research  adopts  a  narrow  view,  not  concerned  with  the 
decision  process,  but  concerned  with  the  logic  of  choice.  He  discusses  Attaway's  beliefs 
on  the  difficulties  of  linking  the  objective  to  an  effectiveness  scale.  While  effectiveness 
of  something  like  a  “defensive  aircraft  interceptor  force”  is  highly  sensitive  to  the  chosen 
scale  (enemy  destroyed,  friendly  survivors,  etc.),  the  only  guidance  for  building  the  scale 
is  to  reflect  the  problem  while  making  the  measurement  as  simple  as  possible.  Keen 
states: 

As  problems  move  up  the  scale  of  complexity-and  multidimensionality-from 
operational  analysis  tu  tactical  and  then  strategic  planning,  this  linkage 
becomes  almost  intractable  .  .  .The  actual  choice  of  a  decision  necessarily 
requires  subjective  appraisal.  (1977:37) 

To  overcome  these  difficulties,  methods  such  as  Delphi  have  been  to  developed  to 
synthesize  consensual  scales  .  Keen  suggests  complex  planning  is  “essentially  marked  by 
multiple  criteria  and  that  decision  makers  are  more  concerned  with  reducing  risk, 
cognitive  strain  and  conflict  than  with  optimizing  solutions.  '  Approaches  to  Niulli- 
Criteria  Decision-making  (MCDM),  goal  programming,  and  compromise  programming 
attempt  to  “mesh  the  efficiency  and  logic  ot  optimizatUm  with  the  decision  maker's 
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emphasis  on  satisficing.”  Keen  discusses  how  goal  programming,  with  its  roots  in  linear 
programming,  has  not  been  successfully  implemented  in  many  situations.  This  lack  of 
success  contrasts  w  ith  the  success  of  a  hybrid,  interactive  methodology  that  “has  become 
consensually  accepted  as  the  basis  for  formal  decision  aids  for  MCDM.”  Instead  of 
utility  measurements  and  relative  weights,  decision  makers  are  presented  outcomes  and 
asked  to  provide  tradeoff  information.  Various  methods  of  coding  the  tradeoffs  have 
been  proposed  (marginal  rate,  ordinal  rankings,  better  or  worse).  Their  results  provide  a 
situational  utility  function.  The  hybrid  methodology  shifts  the  emphasis  from  choice  to 
“the  definition  of  relevant  alternatives”  and  leaves  choice  to  the  decision  maker's 
subjective  judgement.  Keen  believes  it  is  too  early  to  judge  how  effective  hybrid 
methods  will  be,  but  they  do  pass  the  test  of  being  behaviorally  grounded,  a  test  the 
rational  model  can't  pass  for  most  decisions  (Keen,  1977:3 1-57).  Bernard  Roy  and 
Thomas  Saaty  provide  prominent  approaches  to  Multiple  Criteria  Decision  Making 
(Roy,  1990:324  330;  Saaty,  1990:5-34). 

One  of  the  new  theories  in  decision-making  is  the  garbage  can  model.  It  is 
essentially  an  offshoot  of  rational  decision  theory  which  considers  decision-making  undei 
conditions  of  ambiguity.  It  was  initially  used  to  describe  academic  decision-making  and 
has  been  expanded  to  other  public  bureaucracies  such  as  the  military.  The  garbage  can 
model  provides  an  important  transition  from  the  strict  rational  models  to  the  muddled 
descriptive  models  (Murray,  1986:31).  It  is  important  because  it  addresses  limitations 
that  have  been  criticized  in  the  rational  model. 

Roger  Weissinger-Baylon  discusses  the  application  of  Cohen  et  al.'s  garbage  can 
modt;l  to  naval  decision-making  in  a  paper  presented  at  the  Naval  Postgraduate  School's 
1984  conference  on  military  decision-making.  The  garbage  can  model  is  applicable  to 
organizational  decision-making  with  ambiguity  in  three  areas:  technology,  preferences, 
and  participation.  Ambiguity  in  technology  means  organizations  don't  understand  their 
processes.  Ambiguity  of  preference  means  the  decision  maker  preferences  are  unclear  or 
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changing.  Ambiguity  of  participation  means  decision  makers  move  in  and  out  of 
“decision  arenas.”  While  Weissinger-Baylon  doesn't  conclude  that  the  military  totally 
reflects  the  “organized  anarchy"  that  Cohen  attributed  to  organizations  that  experience  all 
three  ambiguities,  he  does  show  how  the  model  can  provide  significant  insights  not  found 
in  the  normative  or  rational  decision  models.  Weissinger-Baylon  draws  his  conclusions 
based  on  observations  at  a  single,  week-long  war  game  at  the  Naval  War  College  and 
additional  sources  of  information  (interviews  with  30  flag  officers  and  others,  and 
observations  aboard  two  U.S.  Navy  ships).  He  found  the  Navy  mitigated  the  “organized 
anarchy”  that  should  flow  from  its  decision-making  ambiguities  by  using  Standard 
Operating  Procedures,  operational  plans,  and  strong  centralized  authority.  The  Navy 
retained  tight  control  and  coordination  despite  the  ambiguities  involved  in  using  these 
techniques  (Weissinger-Baylon,  1986:37-51). 

Joseph  Wohl  developed  a  taxonomy  of  Air  Force  Tactical  Command  and  Control 
(C^)  force  management  decisions  and  related  these  decisions  to  decision  theories  and  to 
decision-aiding  programs.  He  also  discussed  the  issues  related  to  providing  automated 
assistance  to  these  decisions.  Wohl  seeked  to  energize  C^  system  designers  to  realize 
that  decision  makers,  instead  of  “things,”  are  the  central  issue  in  C^  systems.  The  basic 
decision-making  process  issues  Wohl  described  can  be  summarized  by  saying  that 
automation  is  being  driven  into  the  processes  because  the  speed  of  warfare  has  increased 
data  rates  to  levels  beyond  unassisted  human  information  handling  capacity.  He 
categorized  the  needed  decisions  into  two  categories:  1 )  planning  and  commitment  ■ 
preflight,  command-oriented  decisions,  and  2)  control  and  coordimtion  -  alert/inflight, 
control,  time-critical  decisions.  Each  poses  different  issues.  Unit-level  mission  planning 
is  included  in  the  planning  and  commitment  decision  category.  Wohl’s  review  of 
research  related  to  the  tactical  decision-making  process  concluded  there  has  been  little 
agreement  on  characterizing  a  model  of  the  human  decision  process  (Wohl,  1991:  13). 
Wohl’s  analysis,  based  on  interviews  with  Air  Force  personnel  in  the  early  1980s, 
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concluded  that  uiiii-level  detailed  planning  required  decision-aiding  in  the  areas  of 
“Stimulus”  (gathering  and  correlation  of  data),  ge»'erating  and  evaluating  “Options,”  and 
selecting  a  “Response.”  However,  his  research  indicated  decision  aids  were  being 
developed  only  in  the  “Stimulus”  and  “Response”  areas  (Wohl,  1991:  28).  Wohl  cautions 
that,  despite  the  potential  for  improving  unit-level  planning  with  decision  aids, 
“information  uncertainty,  information  overload,  and  time  available  for  decision-making 
will  impact  their  utility.”  He  suggests  fundamental  research  into  how  the  behavioral 
factors  interact,  how  they  are  affected  by  decision  aids,  and  how  aided  decisions  impact 
force  effectiveness  (Wohl,  1991:31).  We  review  behavioral  issues  in  our  next  section. 

Behavioral  Factors.  The  theory  of  rational  choice,  commonly  called 
decision  theory,  has  three  basic  assumptions  or  principles:  1 )  decision-makers  can  decide 
what  they  prefer,  2)  preferences  are  transitive,  and  3)  decision-makers  always  prefer  more 
utility  to  less,  However,  experimental  evidence  indicates  people  often  do  not  apply  these 
principles.  For  example,  the  principle  of  transitivity  implies  that  if  "Job  A  is  preferred  to 
Job  C,  and  Job  C  to  Job  D,  then  Job  A  is  preferred  to  Job  D."  Experimental  evidence 
indicates  the  process  of  binary  comparisons  often  leads  to  inconsistent  choices  instead  of 
the  consistency  inferred  by  the  transitivity  principle.  There  are  several  other  examples  of 
people  violating  the  principles  of  rational  choice  that  they  willing  accept  when  they  are 
stated  abstractly.  These  behavioral  issues  do  not  mean  that  the  principles  are  lacking  but 
that  there  are  behavioral  factors  decision  analysts  should  consider.  Three  prominent 
behavioral  sources  of  sub-optimal  choice  are:  “1)  the  emotional  state  of  the  individual,  2) 
the  manner  in  which  the  choice  situation  is  structured,  and  3)  human  limitations  in 
processing  information.”  These  sources  interact  in  affecting  actual  choices  (Hogarth, 
1983:64-72). 

A  1981  article  by  Andrew  Sage  has  been  cited  as  “the"  comprehensive  review  of 
decision,  behavioral,  and  psychological  theories  relevant  to  the  designers  of  decision 
support  processes  and  systems.  Sage’s  39-page  article  cites  415  sources  in  its  discussion 
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of  behavioral  and  organizational  considerations  for  decision  support.  Sage  contends  that 
decision  support  systems  cannot  be  successful  without  considering  the  contributions  of 
several  disciplines.  His  objective  is  to  provide  a  system  engineering  perspective  for 
integrating  the  contributions  of  the  multiple  disciplines  which  can  contribute  to  decision 
support. 

Sage  considers  five  different  areas:  1 )  cognitive  styles,  2)  human  information 
processing  in  decision  situations,  3)  decision  rules  for  individuals,  4)  contingency  task 
structural  models,  and  5)  decision-making  frameworks  in  organizational  settings  (Sage, 
19191:31.5).  From  the  review  of  cognitive  styles  material,  he  concludes  that  while 
cognitive  styles  are  an  important  consideration  in  decision  support  they  are  not  an 
accurate  predictor  of  decision  performance.  The  prime  determinant  of  human  behavior  is 
not  style  but  task  characteristics  (318).  From  his  review  of  information  processing 
literature.  Sage  concludes  that  the  literature  tends  to  support  Slovic's  hypothesis  that 
^'humans  may  well  be  little  more  than  masters  of  the  art  of  self  deception.  He  suggests 
that  while  this  limits  the  ability  to  improve  human  Judgement,  it  increases  the  need  to 
focus  on  tools  which  assist  in  acquisition,  analysis  and  interpretation  of  information 
(323).  Sage  suggests  the  decision  niles  research  is  somewhat  inconclusive,  with  the 
strongest  evidence  supporting  the  thought  that  choice  of  decision  rules  is  very  task- 
dependent  a.id  appreciably  varies  with  various  interpretations  of  the  same  situation.  Sage 
cites  several  on-going  research  efforts  which  are  searching  for  more  conclusive  results 
(.*130).  In  his  final  section,  Sage  discusses  various  frameworks  for  rational  group 
decision-making  and  how  support  systems  can  be  applied  to  detect  and  conect  cognitive 
biases  (336-34.5), 

Sage's  analysis  is  impressive  because  of  Us  comprehensiveness  and  numbing  in  its 
depth.  Sage  suggests  several  methods  of  how  information  systems  can  be  used  to 
structure  decision-making  so  that  behavioral  bias  can  be  detected  and  possibly  corrected. 
Hogarth  best  summarizes  the  situation: 
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it  is  important  to  emphasize  that  human  judgemental  activity  is  proficient  in  a 
wide  range  of  demanding  tasks.  However,  judgemental  abilities  have 
developed  in  circumstances  of  relatively  slow  evolutionary  change.  As 
increasing  technological  development  acts  to  accelerate  change,  the 
deficiencies  of  human  judgement  are  inevitably  accentuated.  Nonetheless, 
technology  can  be  adapted  to  complement  human  judgement  lather  than 
expose  its  weaknesses.  However,  to  accept  such  aids,  people  must  first  be 
willing  to  admit  theli  own  deficiencies.  (Hogarth,  19^:1^) 

A  summary  of  the  behavioral  factors  or  biases  in  information  processing  discussed  by 
Hogarth  is  presented  in  I'able  2-1.  Both  Sage's  and  Hogarth's  research  suggests  that 
despite  the  behavioral  imperfections  of  the  rational  model,  it  can  still  be  the  basis  of 
analysis  if  other  factors  arc  also  considered. 

Dccision-Aidinp.  While  the  introduction  to  this  thesis  noted  that  the  Air  Force 
realized  the  need  to  reconsider  the  structure  of  the  ACMi  ’  process  while  automating  it, 
the  Air  Force  is  nevertheless  depending  on  automatic  decision- ??*ding  to  increase  planning 
effectiveness.  The  conclusions  from  our  review  of  operational  literature  noted  the  trend 
t(>v/aid  increased  reliance  on  technology  to  keep  pace  with  the  expanding  planning 
workload.  However,  our  inability  to  assume  that  decision-aiding  will  improve 
effectiveness  requires  we  understand  decision-aiding.  Out  review  considers  both  the 
theory  supporting  decision-aiding  and  methods  for  evaluating  decision-aiding. 

Theory.  Paul  Lehner  contends  concepts  a’oout  automated  decision 
support  come  from  three  academic  areas.  First,  the  concept  of  decision  aids  that  assist 
decision  makers  in  structuring  decisions  to  conform  to  theoretical  ideals  of  rational 
judgement  comes  from  the  human  judgement  and  decision-making  community.  Second, 
the  idea  of  Decision  Support  Systems  (DSS)  that  provide  advice  based  on  quantitative 
analysis  comes  from  information  systems.  Third,  the  idea  of  providing  decision  makers 
an  expert's  approach  throui^h  a  computer  system  comes  from  the  expert  systems 
community.  While  each  of  these  areas  has  identified  factors  that  impact  decision  support 
quality,  Leliner  states  that  use  of  the  factors  does  not  provide  reliable  enginccrir« 
principles  for  developing  systems.  Based  on  his  analysis  of  tlie  factors,  Lehntr  proposes 
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Processing  of  Iiifaniiatktn  I  Acquisition  of  Informatior 


Table  2-1 A 

Behavioral  Factors  In  Information  Processing 

(adapted  from  Hogarth.  1983: 166-170) 


Factor  (Source  of  Bias) 

Description 

Availability 

-  with  which  specific  instance.*:  can  be  recalled  from 
memory'  affects  judgements  ul  rrequency. 

-  Chance  'avaiiability'  of  particular  'cues'  in  the  immediate 
enviixMunent  affects  judgement 

Selective  Perception 

-  People  sLTicture  problems  on  the  basis  of  their  own  cxpurience. 

-  Anticipatic  ;!S  of  w  hat  one  expects  (o  see  biases  what  one  does 
see. 

-  People  seek  information  consistent  with  their  ow'n 
views/h^potheses. 

•  People  downplay/disregurd  c<  >nl1iciing  ev  idenue. 

»quency 

1 

-  Cue  used  to  judge  sircp;*t|.  oi  predictive  relationships  is 
obst'oed  fTcqucncy  rat'.i;.i  thaii  obseoed  telativc  frequency. 
Infoi  matiott  on  'non-occurrvnocs'  oi'  an  event  is  often 
unavailable  and  frequently  ijmored  w  hen  avui.iuble. 

Concrete  information 
(ignoring  base-rate  or 
prior  information) 

-  Cortf  rele  infonnahon  (i.c,,  vivid  or  based  on 
e.xperience/irtcidcqts)  dominate.s  abstract  information  (c.g., 
summari  u,  statistiad  base-rates). 

lUusoty  correlation 

1  ...  - 

-  Belief  that  two  varablos  covaiy  when  in  fact  they  do  not 
(Possibly  reltf-ted  to  'ft  equeney'  above). 

Data  Presentation 

-  Order  effects  (primacy/rcccncy). 

•  Mode  of  presentation. 

•  Mixture  of  types  of  information,  e.g  ,  quulitiative  and 
quantitative. 

-  Logical  dam  displays. 

-  Context  efiecLs  pereciv  ed  vanability. 

Inconsistency 

-  Inability  to  apply  a  consistent  judgemental  strategy  over  a 
repetitive  set  of  cases. 

Conservatism 

-  Failure  to  revise  opinion  i/n  receipt  of  new  information  lo  the 
same  exetent  as  Bayes'  theorem. 

Non-linear  extrapolation 

-  Inability  toe.\Uapolate  growth  processes  and  tendancy  to 
underestimate  joui It  prubabiliiies  of  several  events. 

Heuristics  used  to  reduce 
'mental  effort: 

-  Habit/niles  of  thumb 

1 

-  Anchoring  and 
adjustment 

-  Representativeness 

-  Choosing  on  alternative  because  it  has  previously  been 
salisfuctoiy. 

•  Prediction  made  by  anchonng  on  a  clue  or  value  and  then 
adjusting  to  allow  for  the  circumstances  of  the  present  case. 

-  Judgement  of  likelihood  of  an  event  by  estimating  degree  of 
similarity  to  'he  class  of  events  of  w  hich  it  is  supposed  to  be  an 
exemplar. 
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Feedback  I  Ootpat  I  Processing  of  lafi 


Table  2-lB 

Behavioral  Factors  In  Information  Processing  (Continued) 


( adapted  from  Hogarth,  1983:166-170) 


Factor  (Source  of  Bias) 

Description 

-  Law  of  small  numbers 

•  Jusdnability 

-  Regression  bias 

.  'Best-guess'  strategy 

-  Characteristics  of  small  samples  are  deemed  to  be 
representative  of  populations  from  which  they  are  drawn. 

-  A  'processing*  rule  can  be  used  if  the  individual  finds  a 
rationale  to  'justify*  it 

-  Extreme  values  of  a  variable  are  used  to  predict  extreme  values 
of  the  ne.xt  observation  of  the  variable  (thus  failing  to  allow  for 
regression  of  the  mean). 

-  Under  conditions  invoiviug  several  sources  of  uncertainty, 
simplirication  is  made  by  ignoring  some  uncertaintites  and 
basing  judgement  on  the  'most  likely'  hypothesis.  More 
generally,  tendency  to  discount  unccftaint>'. 

The  decision  environment: 

-  Complexity 

-  Emotional  stress 

-  Social  pressures 

-  Complexity  induced  by  time  pressure,  information  overload, 
distractions  leads  to  reduced  consistency  of  judgement. 

-  Emotional  stress  reduces  the  care  with  which  people  select  and 
process  information. 

-  Social  pressures,  e.g.,  of  a  group,  cause  people  to  distort  their 
judgements. 

Information  Sources: 
-Consistency  of 
information  sources 

•  Consistency  of  information  sources  can  lead  to  increases  in 
confidence  in  judgement  but  not  to  increased  predictive 
accuracy. 

Response  Mode: 

•  Question  format 

•  Scale  effects 

•  The  way  a  person  is  required  or  chooses  to  make  a  judgement 
can  affect  the  outcome. 

•  The  scale  on  which  responses  arc  recorded  can  affect 
responses. 

Wishful  thinking 

•  People's  preferences  for  outcomes  of  events  affect  their 
assessment  of  the  events. 

Illusion  of  control 

-  Activity  concerning  an  uncerluiit  outcome  can  by  itself  induce 
in  a  person  feelings  of  control  over  the  uncertain  event. 

Outcome  irrelevant 
learning  structures 

-  Outcomes  observed  yield  inaccurate  or  incomplete  information 
concerning  predictive  relationships.  This  can  lead,  inter  alia, 
to  unrealistic  confidence  m  one's  own  judgement. 

Misperception  of  chauce 
fluctuations  (e.g., 
gambler's  fallacy) 

•  Obsersation  of  an  unexpected  number  of  similar  chance 
outcomes  leads  to  the  expectation  that  the  probability  ol'  the 
apperance  of  on  event  not  recently  seen  increases. 

Success/failure 

attributions 

-  Tendency  to  attribute  succes  to  one's  skill,  and  failure  to 
chance. 

Hindsight  bias 

-  In  retrospect,  people  are  not  'surprised'  about  what  has 
happened  in  the  past.  They  con  easilv  find  plausible 
explanations. 
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two  valid  measures  of  decision  support  quality-robustness  and  substantial  effect. 
Robustness  is  consistency  and  substantial  effect  means  the  decision  support  causes  a 
large  increase  in  the  probability  of  success.  He  makes  his  approach  practical  by 
providing  methods  to  measure  these  two  factors  with  small  sample  sizes  (Lehner, 
1989:273-278). 

Kenneth  Hammond  applies  ideas  from  the  first  of  Lehner's  three  academic  areas, 
human  judgement  and  decision-making,  to  dynamic  tasks.  Hammond's  ideas  are 
important  for  the  dynamic  environment  of  military  decision-making.  Hammond,  like 
Sage  and  Hogarth,  believes  information  systems  can  influence  the  structure  of  the 
decision-making  process.  Hammond  explains  how  the  decision  maker's  use  of  either 
intuition  or  analysis  for  decision-making  can  be  induced  by  how  information  systems 
present  tasks.  Yet,  intuition  and  analysis  have  characteristics  which  may  or  may  not 
make  them  appropriate  to  the  needed  decision  (Hammond,  1988:3-0). 

Stephen  Andriole  reviewed  decision  support  advances  in  the  second  of  Lehner's 
three  academic  areas,  information  systems.  He  mentions  the  current  situation  in  military 
decision  support  systems  where  users  must  manually  integiate  information  from  different 
systems.  He  attributes  this  lack  of  automated  integration  not  only  to  procurement 
craziness,  but  also  to  past  concepts  of  decision  support.  He  predicts  future  decision 
support  capabilities  will  be  embedded  in  larger  integrated  organizational  systems. 
Additionally,  he  suggests  that  interactive  decision  support  will  fundamentally  change 
with  incorporation  of  new  technologies.  Decision  support  wiil  evolve  from  only 
supporting  data  manipulation  to  the  complex  use  of  models  and  hardware  to  support 
option  generation,  evaluation,  and  selection  (Andriole,  1989:254-256). 

Evaluation.  Decision  .Support  Systems  (DSS)  have  become  increasingly 
popular  over  the  last  ten  years  as  more  applications  have  become  available  and  their  costs 
have  been  reduced.  They  are  the  most  prominent  means  of  providing  automated 
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decision-aiding.  However,  the  effectiveness  of  D3S  remains  unproven.  Investments  in 
DSS  are  based  on  cost-effectiveness  analysis  that  assume  DSS  will  increase  the 
effectiveness  of  decisions  and/or  the  efficiency  of  the  decision  process.  These 
investments  are  often  supported  with  anecdotal  results  of  other  applications.  However, 
while  these  increases  in  effectiveness  and/or  efficiency  may  seem  intuitively  logical,  the 
results  of  experimental  studies  are  mixed  (Sainfort  et  al.,  1990:233;  Sharda  et  al., 

1988: 139).  Despite  the  norm  of  evaluating  the  effectiveness  of  investments,  systematic 
evaluation  of  DSS  is  the  exception  rather  than  the  rule.  It  is  estimated  that  reliable 
evaluations  are  conducted  on  only  one  in  ten  government-sponsored  DSS.  Further,  the 
evaluations  are  often  “perfunctory  studies  tacked  onto  tlie  development  cycle”  (Riedel 
andPitz,  1986:980). 

Major  DSS  Evaluation  Studies.  Despite  the  difficulties  involved  in 
measuring  DSS  effectiveness,  there  have  been  several  attempts.  Sharda  et  al,  conducted 
an  extensive  review  of  DSS  effectiveness  research.  They  found  the  mtyority  of  results 
are  based  on  case  or  field  studies  which  they  note  lack  the  experimental  controls 
necessary  for  drawing  strong  inferences  of  DSS  benefit.  They  summarize  the  results  of 
three  field  tests  and  eleven  laboratory  studies  that  have  stronger  experimental  controls. 
These  results  provide  mixed  results  on  whether  DSS  improve  decision-making.  Sharda  et 
al.  used  these  results  as  the  basis  for  designing  a  laboratory  study  measuring  decision 
improvement  in  groups  of  college  students  playing  a  management  game  that  they  argue  is 
an  improved  experimental  design  over  previous  studies.  They  conclude  that  groups  using 
the  DSS  reached  better  decisions  in  approximately  the  same  amount  of  time  after  a  short¬ 
term  “learning  curve”  loss  of  efficiency.  Because  the  majority  of  earlier  experiments  that 
did  not  show  DSS  increases  in  effectiveness  had  only  one-time  measures,  (hey 
hypothesize  those  results  may  have  been  distorted  by  “learning  curves.”  They  suggest 
effectiveness  is  best  measured  with  studies  over  time  using  “longitudinal  designs” 

(Sharda  ef  a/.,  1988:140-156). 
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Sainfort  et  al.  propose  a  conceptual  framework  for  DSS  effectiveness  evaluation 
that  measures  more  than  outcomes.  They  argue  that  for  each  specific  problem  it  is 
difficult  to  determine  the  contribution  of  the  DSS  to  the  final  outcome  because  “it  is 
almost  impossible  to  know  what  the  same  problem  solver  in  the  same  situation  would 
have  done  without  the  DSS."  They  propose  to  evaluate  DSS  effectiveness  by  looking  at 
the  decision  process  based  on  the  assumption  that  assistance  in  phases  of  the  problem¬ 
solving  process  may  increase  the  likelihood  of  solving  the  problem.  They,  like  Sharda  et 
at.,  also  make  assumptions  about  the  problem  solving  process  because  there  is  no 
universally  accepted  or  empirically  superior  model  of  the  problem-solving  process.  They 
used  their  framework  to  conduct  a  laboratory  study  of  fifty-one  couples  seeking  to 
resolve  various  conflicts  between  the  partners.  Twenty-one  couples  used  a  conflict 
resolution  DSS,  twenty  used  a  video-tape  based  contlict  resolution  process,  and  a  ten- 
couple  control  group  was  not  assisted.  The  results  showed  that  while  decision  outcomes 
were  not  significantly  improved  by  the  DSS  as  compared  with  the  video-tape,  the  DSS 
participants  scored  higher  in  two  process  measures  (number  of  alternatives  generated  and 
perceived  progress  in  resolution)  ( 1990:233-250). 

Riedel  and  Fitz  propose  a  "Utilization-Oriented  Evaluation"  of  DSS  in  which  the 
evaluation  design  is  driven  by  the  intended  use  of  evaluation  results.  They  present  broad 
outlines  of  a  model  for  tailoring  Measures  Of  Effectiveness  (MOEs)  based  on  the 
intended  use.  The  broad  model  avoids  the  specificity  of  either  the  Sharda  or  Sainfort 
approaches.  They  review  several  different  evaluation  models,  arguments  for  objective 
instead  of  subjective  measures,  arguments  that  process  measures  are  superior  to  outcome 
measures,  and  other  issues.  They  provide  a  description  of  the  Adelman  and  Donnell 
framework  that  has  seven  categories  of  MOEs  and  several  multiattribute  utility 
technology  (MAUT)  evaluation  models.  They  argue  that  while  authors  of  differing 
evaluation  methods  imply  a  priori  to  have  the  best  evaluation  method,  these  judgements 
can  be  made  only  within  the  context  of  the  evaluation  purpose  and  situation.  They  argue 
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that  the  place  to  start  from  when  selecting  measures  of  effectiveness  (MOEs)  is  the 
management  decision  being  supported  instead  of  an  expert  weighting  of  attributes: 

Evidence  exists  that  selecting  attributes  (or  MOE’s)  for  a  MAUT  evaluation  by 
having  an  expert,  or  someone  who  is  to  use  the  results  of  the  evaluation,  assign 
weights  to  a  generic  set  of  attributes  may  be  a  questionable  procedure.  Wooler 
discussed  this  issue  with  respect  to  career  decision  aids.  He  reports  that  asking 
user  to  weight  a  preselected  list  of  attributes  does  not  produce  the  same 
attribute  set  as  when  users  generate  their  own.  (Riedel  and  Pitz,  19^:982) 

While  Riedel  and  Pitz  suggest  MAUT  can  be  helpful,  they  argue  MAUT  should  be  used 
only  within  a  comprehensive  evaluation  framework  that  includes  modeling  the 
management  problem.  Riedel  and  Pitz  provide  a  comprehensive  framework  for  DSS 
evaluation.  However,  Riedel  and  Pitz  acknowledge  their  framework’s  disadvantage  is  the 
heavy  burden  it  places  on  the  evaluator.  It  assumes  a  technically  competent  evaluator 
with  wide  knowledge  of  the  extensive  evaluation  options  and  factors  which  span  many 
disciplines.  They  assert  the  need  to  make  this  assumption  to  avoid  limiting  the  usefulness 
of  evaluations.  They  suggest  development  of  a  manual  to  assist  the  evaluator  in 
recognizing  the  issues  and  making  the  evaluation  technique  choices  (Riedel  and  Pitz, 
1986:981-994). 

Coll  et  al.  also  conducted  a  DSS  effectiveness  experiment  on  college  students,  but 
their  research  identifies  significant  variables  not  included  in  earlier  studies.  The  students 
were  divided  into  three  groups,  one  using  a  DSS,  one  using  a  structured  manual  process, 
and  one  an  unstructured  process  control  group.  Each  group  was  given  three  decision 
alternatives  and  four  decision  criteria.  The  researchers  hypothesized  DSS  group  members 
would  take  longer  to  decide  but  would  make  better  decisions.  However,  while  the  DSS 
group  did  take  longer  to  decide,  its  decisions  were  not  better  than  either  of  the  two  other 
groups.  Faced  with  results  that  were  not  rational,  Coll  et  al.  propose  factors  that  might 
explain  the  results.  First,  they  go  back  to  behavioral  factors  an  decision-making. 
Specifically,  that  under  pressure  decision-making  is  “emotive,  intuitive,  and  only  partially 
cognitive  and  many  models  suggest  better  information  has  negligible  impact.”  Second, 
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they  turn  to  the  psychology  related  to  computer  use  and  suggest  their  students  were 
computer  reluctant  and  not  responsive  to  the  training  they  were  provided.  They  conclude 
that  their  most  important  finding  is  that  “If  a  climate  of  felt  need  is  lacking,  DSS 
availability  will  probably  not  enhance  decision-making  activity”  (Coll  etai,  1991:77-81). 

Evans  and  Riha  suggest  using  the  “Evaluation  Research”  methodology  that  was 
developed  in  the  social  sciences  to  evaluate  quantitatively  the  effectiveness  of 
government,  social,  and  education  programs.  While  their  methodology  is  tailored,  it 
closely  follows  Sprague  and  Carlson's  general  evaluation  methodology.  Evans  and  Riha 
make  a  strong  case  that  this  methodology  is  applicable  in  evaluating  DSS  from  the 
“standpoint  of  the  contribution  the  system  makes  to  achieving  organizational  goals  and  its 
performance  in  the  operational  environment.”  They  demonstrate  its  application  by 
discussing  a  case  in  which  they  applied  it  to  a  Nevada  hazardous  waste  DSS.  While  their 
methodology  addresses  efficiency  issues,  speed  of  decisions  and  cost  of  the  decision¬ 
making  process,  they  focus  on  effectiveness.  Effectiveness  addresses  whether  the 
organization  is  making  better  decisions  with  the  DSS.  The  nine  steps  in  the  methodoiogy 
proceed  from  identifying  what  to  evaluate  to  a  feedback  step  of  evaluating  the  evaiuation 
process.  Evans  and  Riha  argue  that  by  using  the  extensive  collection  of  Evaluation 
Research  toois,  DSS  evaiuation  can  move  from  subjective,  perception  based  evaiuation  to 
reaiistic,  objective  evaiuation  (Evans  and  Riha,  1989:198-205). 

Group  Decision-Makin£  /  Group  Decision  Support  Systems.  An 
emerging  area  of  importance  in  DSS  effectiveness  evaiuation  is  group  decision-making. 
Group  decision-making  requires  consideration  of  social  variables  that  directly  affect  the 
quality  of  decisions.  Examples  include  interaction,  stimulation  and  suppression  of 
individual  creativity,  and  information  transfer  within  the  group  (Evans  and  Riha, 
1989:199)  Groups  make  decisions  in  a  process  that  differs  from  individual  decision¬ 
making.  Groups  usually  develop  protocols  for  the  decision  process  in  hopes  of  reaching  a 
consensus.  Because  of  differences  between  group  and  individual  decision-making.  Group 
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Decision  Support  Systems  (GDSS)  must  differ  from  individual  DSS  (Smith  and  Sage, 
1991:93).  Snyder  et  al,  provide  three  specific  paradigms  for  GDSS  in  environments  like 
that  of  the  military  (Snyder  era/.,  1989:480-481).  Smith  and  Sage  review  previous 
research  that  shows  that  GDSS  can  significantly  improve  group  understanding  of  the 
problem,  but  research  has  not  shown  that  GDSS  improves  the  efficiency,  reliability  or 
quality  of  group  work.  Apparently,  some  implementations  of  technology  can  heighten 
group  conflict  rather  than  improving  decision  support  (Smith  and  Sage,  1991:91-96). 

Analvsis/Conclusions  From  Academic  Review.  Our  literature  review  of 
academic  research  was  an  exploratory  search  of  a  wide  range  of  areat:  that  affect  the 
evaluation  of  ACMP  effectiveness.  This  wide  scope  was  necessary  because  we  did  not 
find  previous  research  that  provided  direct  guidance  for  meeting  our  objectives.  Findings 
from  the  review  that  are  key  to  our  research  are  summarized  in  Figure  2-1 .  We  draw 
several  conclusions  from  these  findings.  The  increasing  complexity  of  unit-level 
decision-making  and  the  compression  of  decision  time  drives  the  need  to  provide 
increasing  amounts  of  automated  decision-aiding.  Improvements  in  computer  technology 
provide  the  Air  Force  the  capability  to  provide  the  needed  support.  However,  one  of  the 
few  areas  of  consensus  in  the  research  is  that  it  can  not  be  assumed  that  providing 
automated  decision-aiding  will  yield  the  best  possible  or  most  effective  ACMP  process. 
Unfortunately,  measuring  effectiveness  is  complicated  by  differing  decision-making 
'nodels  and  wide  uncertainties  about  how  behavioral  factors  interact  in  decision-making 
processes. 


2-43 


_  Academic  Findings _ 

Detailed  unit-level  planning  might  especially  benefit  from  decision  aids  but 
behavioral  factors  will  impact  effectiveness  and  we  are  uncertain  how  the 
factors  interact  (Wohl).  _ _ _ 

Automation  Is  being  driven  into  the  tactical  decision-making  process  because 
the  speed  of  warfare  exceeds  human  information  processing  c^ipacity  (Wohl). 

Decision  Support  Systems  (DSS)  have  become  an  increasingly  popular  way  of 
providing  decision-aiding  but  the  effectiveness  of  using  DSS  remains  unproven 
(Sainfort  eta/.;  Sharda  etal.). _ 

Planning  acts  to  shift  dacisicn-making  to  periods  of  reduced  activity.  Hov^ever, 
timing  remains  critical  and  optimization  of  complex  decisions  isn't  possible 
(Welssinger-Baylon;  Branch). _ 

Group  decision-making  and  support  requires  consideration  of  social  variables  in 
addition  to  Individual  factors.  Aiding  results  are  mixed  (Smith  and  Sage). 

Increasing  technology  can  act  either  to  accentuate  human  judgement 
deficiencies  or  to  complement  judgement.  However,  results  are  Inconclusive 
(Hogarth;  Sage). _ 

There  Isn't  a  consensually  accepted  model  of  decision-making  although  models 
are  generally  grouped  into  rational,  descriptive,  and  derivative  (e.g.  garbage 
can  model)  categories  (Hogarth;  Moskowitz  and  Bunn). _ 

There  Is  no  generally  accepted  or  unifying  theory  (Moll,  Hwang,  Brodsky). 

There  is  a  general  model  for  evaluation  of  that  follows  the  model  of  planned 
experiments  used  In  other  fields  (Sprague  and  Carlson). _ 

Balancing  breadth  and  depth  of  evaluation  is  important  (Sinaiko). 

C2  evaluation  can  be  simplified  by  separately  considering  two  vectors  of 
decision  quality;  generic  (behavioral)  and  mission  (Metersky;  Adelman  etal.\ 
Adelman  and  Ulvila). 

Three  planning  philosophies  are  prominent:  formal,  incremental,  and  systemic. 
ACMP  uses  systemic  v/ith  both  qualitative  and  quantitative  methods  (Saaty  and 
Kearns). 

Most  methodologies  for  evaluating  decision  aid  effectiveness  focus  on  either 
the  overall  or  the  design  deficiency  identification  ends  of  the  evaluation 
spectrum.  Riedel  and  Pitz  propose  a  comprehensive  framework  that  covers 
both  but  it  requires  expertise  oeyond  tiiat  whicli  an  operational  evaluator 
possesses  (Riedel  and  Pitz). 

Research  has  shown  leaving  experts  assign  weights  to  a  generic  set  of 
measures  of  effectiveness  is  a  questionable  procedure  (Riedel  and  Pitz). 


Figure  2-1.  Academic  Findings 


Despite  these  difficulties,  there  are  general  models  for  evaluation  that  can  be 
followed  in  evaluating  the  ACMP  process.  Further,  the  complexity  of  evaluation  can  be 
simplified  by  considering  behavioral  factors  separately  from  other  factors.  Ev  aluation 
can  be  further  simplified  by  focusing  on  either  of  two  ends  of  the  evaluation  spectrum 
(overall  effectiveness  evaluation  or  identification  of  specific  design  deficiencies  that  limit 
effectiveness).  However,  any  simplification  through  focusing  on  only  one  end  of  the 
spectrum  must  be  weighed  against  the  evaluation  requirements.  But,  approaches  that 
attempt  to  be  comprehensive  are  likely  to  be  too  complex  for  use  at  the  operational  level. 
The  research  supports  the  view  that  evaluations  must  be  tailored  to  evaluation  purposes 
for  the  results  to  br  useful.  Until  some  future  breakthrough  occurs,  the  reviewed 
academic  research  indicates  it  is  not  practical  to  design  single  measures  of  effectiveness 
that  can  both  provide  an  overall  effectiveness  measure  and  indications  of  what  process 
adjustments  will  improve  decision  support.  The  Evans  and  Riha  “Evaluation  Research" 
methodology  has  great  promise  for  overall  effectiveness  analysis,  while  the  more 
traditional  information  systems  approaches  such  as  Sharda's  or  Sainfort's,  arc  able  to 
identify  DSS  design  deficiencies.  Riedel  and  Pitz's  approach  is  comprehensive  but  its 
complexity  makes  it  unsatisfactory  for  operational  use.  Integrating  results  from  the 
reviewed  methodologies  was  our  best  option,  and  our  challenge. 

Literature  Review  Analvsis/Conclusions 

While  the  breadth  of  our  operational  and  academic  literature  reviews  provided 
substantial  assistance  in  completing  our  research  objectives,  our  literature  review  did  not 
reveal  research  that  directly  addre  ssed  our  objectives.  The  first  three  sections  of  the 
review  discussing  the  ACMP  process,  previous  mission  planning  research,  and  lessons 
learned  from  Desert  Storm,  directly  support  our  development  of  a  conceptual  model  of 
the  ACMP  process.  This  research  also  revealed  several  trends  that  parallel  findings 
discussed  in  the  academic  literature.  The  fourth  section  of  liieiature  review  discussing 
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academic  reseach  does  not  reveal  a  breakthrough  evaluation  methodology  that  adequately 
simplifies  all  the  complexities  involved  in  operational  ACMP  evaluation .  We  have 
analyzed  both  the  trends  identified  from  the  operational  review  and  findings  from  the 
academic  review  in  attempting  to  identify  those  which  are  signifi'^ant  to  our  research. 

We  provided  that  analysis  at  the  end  of  each  of  those  sections.  While  we  did  not 
necessarily  expect  to  find  parallels  bettveen  our  operational  trends  and  academic  findings, 
those  trends  are  apparent  in  Figure  2-2.  These  parallels  increase  our  confidence  in  our 
analysis  and  conclusions. 

Several  of  the  operational  and  academic  findings  support  the  requirement  for 
providing  automated  decision-aiding  in  the  ACMP  process.  However,  the  conflicting 
results  from  research  on  Decision  Support  System  effectiveness  suggest  that  rigorous 
evaluation  of  systems  and  processes  is  required  to  ensure  improvements  in  effectiveness. 
We  previously  noted  that  Hopple  specifically  makes  this  recommendation.  The 
operational  trends  and  academic  findings  note  the  increasing  influence  behavioral  factors 
have  on  ACMP  decision  quality.  However,  the  academic  research  on  behavioral  factors 
and  group  decision-making  suggests  that  evaluating  these  factors  is  difficult  because 
researchers  do  not  fully  understand  how  the  factors  interrelate.  Because  of  this  difficulty. 
Metersky  and  others  have  recommended  that  behavioral  factors  be  considered  separately. 
Tbs  academic  findings  indicate  that  the  evaluation  purpose  should  guide  formulation  of 
the  evaluation  methodology.  While  Sinako's  report  on  the  Navy’s  attempts  to  develop  an 
evaluation  methodology  indicates  the  need  to  balance  the  breadth  and  depth  of 
evaluation,  most  of  the  reviewed  evaluation  methodologies  focus  on  a  specific  end  of  the 
evaluation  spectrum.  Some  focus  on  overall  effectiveness  evaluation  while  others 
attempt  to  identify  specific  design  deficiencies  that  limit  effectiveness.  Riedel  and  Pitz's 
decision-aiding  evaluation  approach  is  comprehensive,  but  they  note  their  approach 
requires  significant  expertise  to  apply.  Because  none  of  the  approaches  can  be  directly 
applied  to  our  ACMP  evaluation  problem,  we  must  derive  an  acceptable  approach. 
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111.  Methodology 


Overview 


in  mapping  out  a  method  to  achieve  our  research  goal,  we  defined  three  related 
objectives  as  shown  in  Figure  3-1,  By  design,  each  of  these  objectives  provided  a 


PROBLEM 

OBJECTIVES 


RESUIT 


Figure  3>1.  Research  Summary 


measurable  product  that  we  could  use  as  a  milestone  for  tracking  research  completion. 
With  three  objectives,  there  is  not  one  methodology,  but  actually  a  meta- methodology 
(accomplish  objectives  one,  two  and  three)  and  three  separate  methods-an  appropriate 
one  for  each  research  objective.  Before  describing  the  specific  methods  used  to  achieve 
the  research  objectives,  a  discussion  of  models,  the  model  building  process,  and  model- 
based  problem  solving  is  necessary  to  establish  the  base  from  which  the  methodologies 
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were  derived.  After  addressing  the  results  from  that  literatui^  reviev/,  a  research  method 
for  each  of  our  three  objectives  is  presented. 

Literature  Review  -  Models  and  Modeling 

Models  are  abstractions  or  descriptions  of  an  actual  or  proposed  system  (Pritsker, 
19C6;4).  The  system,  idea,  situation,  policy,  or  phenomena  to  be  deied  is  called  the 
problem  entity  (Sargent,  1991:38).  Models  can  range  tfom  an  exact  representation  of  the 
problem  entity,  such  as  a  scale  or  full  size  physical  model,  to  an  abstract  representation, 
such  as  a  mathematical  formula.  Over  the  years,  researchers  have  classified  models  into 
various  categories.  Some  of  these  categories  and  examples  are  shown  in  Table  1 . 

A  conceptual  model  is  defined  by  Sargent  as  a  “mathematical/logical/verbal 
representation  (mimic)  of  the  problem  entity  developed  for  a  particular  study”  (Sargent, 
199138).  A  more  specific  conceptual  model  is  ihe  computerized  model  thaxSwgtvX 
defines  as  '‘the  conceptualized  model  implemented  on  a  computer”  (Sargent.  :38), 
Conceptual  models  are  developed  through  what  Sargent  refers  to  as  an  analysis  and 
modeling  phase  (Sargent,  1991:38). 

Models  can  be  used  for  vaiious  purposes.  However,  “a  model  should  be 
developed  for  a  specific  purpose  or  application  and  its  validity  determined  with  respect  to 
that  purpose"  (Sargent,  1991:37).  The  model's  purpose  or  objective  is  the  controlling 
factor  in  building  and  validating  the  model.  The  model,  therefore,  is  assrjciated  with  a 
particular  problem,  objecti  ve,  or  set  of  questions. 

In  a  general  sense.  Shannon  states  Uiat  a  model  can  serve  one  of  two  purposes: 
“cither  descriptive,  ior  explaining  and/or  understanding:  ox  prescriptive,  by  predicting 
and/or  duplicating  behavior  characteristics”  (Shannon,  1975:7).  Pritsker  takes  a  more 
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V 


Table  3-1 

Model  Categories  and  Examples 


detailed  view  and  identifies  seven  functional  levels  of  application  or  purpose: 


-  as  explanatory  devices  to  understand  a  system  or  problem: 

-  as  a  communication  vehicle  to  describe  system  o^ration; 

~  as  an  analysis  tool  to  determine  critical  elements,  components  and  issues  and 
to  estimate  performance  measures; 

~  as  a  design  assessor  to  evaluate  proposed  solutions  and  to  synthesize  new 
alternative  solutions; 

-  as  a  scheduler  to  develop  on-line  operational  schedules  for  jobs,  tasks  and 
resources; 

-  as  a  control  mechanism  for  the  distribution  and  routing  of  materials  and 
resources:  and 

-  as  a  training  tool  to  assist  operators  in  understanding  system  operations. 
(Pritskere/a/.,  1991:1201-1202) 

Model-Based  Problem  Solving  Process.  Models  are  excellent  tools  for 
helping  decision  makers  solve  problems.  In  Figure  3-2,  an  approach  suggested  by 
Piitsker  for  solving  problems  using  models  is  illustrated.  Using  this  approach,  the 
modeler  (or  decision  maker)  begins  with  or  defines  a  problem  based  on  the  system  needs 
and  requirements.  After  formulating  the  problem  statement,  data  or  information  about  the 
system  is  collected  and  a  model  (or  models)  is  developed  **10  accordance  with  the 
problem  statement  and  available  system  information”  (Pritsker  e/ a/.,  1991:1201). 
Depending  upon  the  problem  or  number  of  alternatives  to  be  investigated,  several 
versions  of  the  basic  model  may  be  required.  By  experimenting  with  or  analyzing  the 
output  from  the  various  model  versions,  the  modeler  (or  decision  maker)  can  obtain 
information  that  can  be  used  to  solve  the  problem  or  set  policies  that  may  allow  decisions 
to  be  made. 

This  process  is  usually  iterative.  Information  obtained  from  the  model(s)  may 
generate  further  questions  and  the  need  for  more  information.  These  questions  may 
necessitate  further  models  or  experiments  to  provide  the  additional  information.  Thus  the 
modeler  is  continually  building  or  revising  models  in  order  to  provide  information  to 
support  the  decision-making  procesa. 
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Simulation  in  Problem  Solving.  Shannon  defines  simulation  as 
“experimentation  with  a  model  of  a  real  system*’  (Shannon,  1975:  iO).  This  is  a  much 
broader  definition  of  simulation  than  is  normally  used  by  other  experts.  Most  other 
modeling  or  simulation  experts  apply  the  term  simulation  to  the  process  of  “designing  a 
mathemaiical-iogicai  model  of  a  real  system  and  experimenting  with  this  model  on  a 
computer*'  (Pritsker,  1986:6).  For  ihi  i  research  effort,  we  subscribe  to  tlie  broader 
definition  proposed  by  Shannon  and  will  use  tiie  term  computer  simulation  when 
referring  to  experimeotation  done  v.  ith  models  on  e  computer.  As  shown  in  Pritsker  er 
al.'s  model-based  problem  solving  process  (Figure  3-2),  experimenting  with  the  various 
versions  of  our  system  model  to  gain  useful  information  qualifies  as  simulation. 

Before  addressing  how  models  arc  built,  it  is  important  to  know  when  simulation 
should  be  used.  Simulation  has  several  advantages  and  disadvantages  over  eiriier 
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cxf  ;rimentation  with  the  real  system,  or  calculation  of  an  analytical  solution.  Several  of 
these  as  presented  by  Shannon  are  shown  in  Figure  3-3. 


Advantages 

1.  Simulation  can  be  used  when  a 
complete  mathematical  formulation  of 
the  problem  does  not  exist  or  analytical 
methods  of  solving  the  mathematical 
model  have  not  yet  been  developed. 

2.  Simulation  may  be  simpler  to  use 
than  complex  mathematical  solutions,  or 
when  the  analytical  solutions  are  beyond 
the  ability  of  the  available  personnel. 

3.  Simulation  can  provide  a  simulated 
history  of  the  process  over  a  period  of 
time  and  estimate  certain  parameters. 

4.  Simulation  may  be  used  when  it  is 
impossible  or  difficult  to  experiment 
with  or  observe  the  real  sy  stem. 

5.  Simulation  affords  complete  control 
over  time,  and  allows  for  speeding  up  or 
slowing  down  time  at  will. 

6.  The  process  of  simulation  often 
provides  an  education  and  training 
benefit  while  helping  the  developer 
better  understand  the  problem. 


1.  Development  of  a  good  simulation 
model  is  ohen  expensive  and  time 
consuming.  It  also  may  require  a 
modeling  expert. 

2.  A  simulation  can  appear  to  reflect 
accurately  the  real  system,  when  in  truth 
it  does  not 


3.  Simulation  is  imprecise.  This  degree 
of  imprecision  cannot  be  measured. 
Sensitivity  analysis  can  only  partially 
overcome  this  difficulty. 

4.  Simulation  results  are  usually 
numerical  and  their  displayed  precision 
misrepresented.  This  may  allow  for 
more  validity  to  be  attributed  to  the 
results  than  is  justified. 


F'ignre  3>3,  Advantages/Dbadvantages  of  Simulation  (Shannon,  1975:11-14) 


Model  Buildin2  Process.  “Mode!  building  is  a  complex  process  and  in  most 
fields  is  on  art”  (Pritsker,  1986:4).  The  analysis  and  modeling  phase  has  no  prescribed 
set  of  rules  that  will  work  in  all  cases.  At  best,  any  rules  or  steps  can  only  provide  a 
framework  or  approach.  Although  there  is  no  single  approach  to  building  a  conceptual 
model,  most  model  building  frameworks  follow  an  iterative  or  evolutionary  approach 
(Shannon,  1975;  19).  A  framework  for  the  development  of  a  general  model  proposed  by 
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Mihram  is  shown  in  Figure  3-4.  A  description  of  that  framework  is  outlined  in  the 
following  discussion. 


Flgore  3^.  Modeling  Process  (Mihram,  1972:628) 


Modeling  Goals  —  Stage  0.  Mihram  suggests  that  before  beginning 
the  modeling  activity,  the  modeler  should  clearly  state  the  goal  of  the  model  (Stage  0  of 
Figure  3-4).  He  also  suggests  that  the  system's  measures  of  performance  be  clearly 
defined.  This  definition  stage  of  the  modeling  process  is  similar  to  that  found  in  model 
building  frameworks  proposed  by  other  modeling  experts  who  all  suggest  that  the  system, 
measures  of  performance,  and  the  model's  purpose  be  clearly  defined  before  beginning  to 
build  a  model  (Kiviat,  1967:19;  Shannon,  1975:23;  Pritsker,  1986:4-5). 
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The  model's  goal,  purpose,  or  problem  statement  is  of  paramount  importance  to 
the  modeler.  This  point  is  emphasized  as  a  basic  modeling  principle  by  Pritsker;  'The 
problem  or  problem  statement  is  the  primary  controlling  element  in  model-based  problem 
solving”  (Pritsker  e/  a/.,  1991: 1201),  It  is  a  controlling  element  because  it  not  only 
determines  the  system's  boundaries  but  also  establishes  the  level  of  detail  required  by  the 
model  (Kiviat,  1967:13;  Pritsker  et  a/.,  1991:1201).  Further  importance  to  the  model's 
purpose  is  given  during  the  validation  stage  of  the  modeling  process.  “If  the  purpose  of  a 
model  is  to  answer  a  variety  of  questions,  the  validity  of  the  model  needs  to  be 
determined  with  respect  to  each  question”  (Sargent,  1991:37). 

Systems  Analysis  --  Stage  1 .  After  the  goals  of  the  model  have  been 

clearly  defined,  actual  development  of  the  model  can  begin.  Mihram  calls  this  initial 

stage  of  the  model  development  system  analysis  (Stage  1  of  Figure  3-4).  During  the 

systems  analysis  stage,  the  modeler  breaks  the  system  down  either  into  concrete  or 

abstract  entities.  The  modeler  then  further  identifies  the  entity's  attributes  or 

characteristics.  The  system's  boundaries  are  also  identified  and  items  are  sorted  among 

those  residing  in  the  system's  environment  (external  to  the  model)  and  those  residing 

within  the  system.  After  iterative  analysis: 

The  initial  stage  of  a  model's  development  terminates  with  the  delineation  of 
the  1 )  system  boundary,  2)  system  environment,  3)  system  entities,  4)  entity 
attributes  or  system  state  variables,  S)  the  intrinsic  feedback  mechanisms  or 
activities  of  the  system,  and  6)  the  events  (if  any)  inherent  in  the  system's 
behavioral  structure.  (Mihram,  1972:627) 

System  Synthesis  --  Sta2e  2.  The  second  stage  of  model  building 
addresses  the  model's  structure  and  its  implementation  (Mihram,  1972:627).  At  this  stage 
(Stage  2  of  Figure  3-4),  the  modeler  is  concerned  about  what  the  type  of  model  should  be 
developed.  The  modeler  addresses  issues  such  as  whether  the  model  should  be  material 
or  symbolic;  static  or  dynamic;  and  deterministic  or  stochastic.  After  deciding  on  a 
model  type,  the  modeler  attempts  to  map  the  abstract  elements  of  the  system  identified 
during  the  system  analysis  stage  onto  the  chosen  model  structure. 
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The  choice  between  model  types  can  often  be  made  by  comparing  alternatives 
based  on  their  advantages  and  disadvantages.  Comparison  of  model  accuracies, 
implementation  costs,  data  requirements,  data  availability,  maintainability,  cost  of 
experimentation  and  other  factors  can  help  narrow  the  choice.  The  final  structure  of  the 
model  will  be  affected  by  many  factors.  Some  factors,  as  identified  by  Kiviat,  are: 

--  The  purpose  of  the  model. 

-•  The  accuracy  and  precision  required  of  the  output. 

-  The  detail  required  in  the  model  to  achieve  the  required  precision. 

•’  The  assumptions  required  at  the  system  boundaries. 

—  The  assumptions  required  within  the  system  boundaries  for 

—  status  representation 

—  decision  parameters 

—  decision  rules. 

-  The  availability  of  necessary  data.  (Kiviat,  1967: 18-19) 

In  the  end,  the  final  selection  for  model  type  and  structure  should  be  based  on  the 
purpose  of  the  model  and  its  usefulness  to  its  intended  user  or  decision  maker. 

There  are  no  step-by-step  instructions  for  implementing  the  structure  of  the 
model.  Experts  seem  to  agree  that  it  is  a  creative,  evolutionary  process  closer  to  an  art 
than  a  science  (Shannon,  1975:19:  Pritsker,  1986: 1 1).  It  is  helpful  to  understand  that  '‘the 
secret  to  being  a  good  modeler  is  recognizing  the  need  and  having  the  ability  to  remodel" 
(Pritsker  et  al.,  1991: 1200).  A  successful  approach  for  building  a  complex  model  is  to 
begin  with  a  simple  model,  and  embellish  it  or  enrich  it  (Shannon,  1975:20;  Pritsker  et 
al.,  1991:1205;  Banks  and  Carson,  1984:13).  An  excellent  summary  of  this  concept  of 
system  synthesis  is  provided  by  Banks  and  Carson: 


The  art  of  modeling  is  enhanced  by  an  ability  to  abstract  the  essential  features 
of  a  problem,  to  select  and  modify  basic  a  sumptions  that  characterize  the 
system,  and  then  to  enrich  and  elaborate  the  model  until  a  useful  approximation 
results.  Thus,  it  is  best  to  start  with  a  simple  model  and  build  toward  greater 
complexity.  However,  the  model  complexity  need  not  exceed  that  required  to 
accomplish  the  purposes  for  which  the  model  is  intended.  (Banks  and  Carson, 
1984:13) 


Verification  and  Validation  -  Stages  3  and  4.  During  the 
verification  stage  (Stage  3  of  Figure  3-4)  the  modeler  evaluates  the  model  to  see  if  it  was 
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built  or  developed  as  intended  (Mihranm,  1972:627).  Validation  (Stage  4  of  Figure  3-4)  is 
different  from  verification  in  that  the  modeler  tests  the  agreement  of  the  model's  behavior 
with  the  behavior  of  the  actual  system  (Mihram,  1972:628). 

Consider  the  relationships  depicted  in  Figure  3-5  between  the  problem  entity,  the 
conceptual  model,  and  the  computerized  model.  In  this  figure,  a  conceptual  model  has 
been  developed  by  the  modeler  for  a  problem  entity.  This  conceptual  mode),  which 
exists  in  the  mind  of  the  modeler  or  as  represented  by  a  computer  program  specification, 
represents  the  image  of  the  problem  entity  that  the  modeler  intends  to  model.  Mow 
closely  that  conceptual  model  represents  the  actual  system  or  problem  entity  is  what 
Sargent  refers  to  as  conceptual  model  validity  (SargtnU  1991:39).  When  assessing 
conceptual  model  validity,  the  modeler  confirms  the  “theories  and  assumptions 
underlying  the  conceptual  model  are  correct  and  that  the  model  representation  of  the 
problem  entity  is  'reasonable*  for  the  intended  purpose  of  the  model”  (Sargent,  1991 :39), 


Figure  3*5.  Modeling  Process  Relationships  (Sargent,  1SI91 :39) 
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Once  the  modeler  has  finished  the  programming  effort,  a  computerized  model 
exists.  Ensuring  that  the  computer  programming  and  implementation  of  the  conceptual 
model  are  correct  is  referred  to  as  computerized  model  verification.  A  check  of  the 
programmed  model's  output  accuracy  against  the  problem  entity  is  referred  to  as 
operational  validity. 

A  final  classification  of  validity  depicted  in  the  figure  by  Sargent  is  data  validity. 
“Data  validity  is  defined  as  ensuring  that  the  data  necessary  for  model  building,  model 
evaluation  and  testing,  and  conducting  the  model  experiments  to  solve  the  problems  are 
adequate  and  correct"  (Sargent,  1991:39).  We  see,  therefore,  that  in  each  case  above, 
validity  is  always  a  comparison  between  an  aspect  of  the  model  and  reality.  Verification, 
on  the  other  hand,  is  a  comparison  between  the  progranmied  model  and  the  modeler's 
concept  of  the  system. 

In  the  realm  of  computer  or  progranmied  models,  verification  is  more  than  just 
debugging  the  program.  The  modeler  must  realize  “that  errors  may  be  caused  by  the 
data,  the  conceptual  model,  the  computer  program,  or  the  computer  implementation" 
(Sargent,  1991:41).  For  those  reasons,  there  are  many  different  ways  to  verify  a  model. 
Whitner  and  Balci  have  classified  programmed  model  verification  techniques  into  six 
major  categories:  informal,  sutic,  dynamic,  symbolic,  constraint,  and  formal  (Whitner 
and  Balci,  1989:560).  Each  of  these  major  categories  has  many  specific  techniques  that 
provide  the  modeler  with  unique  advantages  and  disadvantages. 

It  is  important  to  remember  that  the  question  of  validity  must  be  asked  with  regard 
to  a  particular  purpose  or  set  of  questions  that  the  model  is  supposed  to  address.  It  is 
possible  that  a  model  developed  for  two  purposes  may  be  valid  for  one  purpose  and  not 
the  other.  Generally  speaking,  a  direct  comparison,  based  on  experiments  with  the  model 
and  the  real  system,  is  the  preferred  means  of  determining  model  validity.  Unfortunately, 
that  is  not  always  possible. 
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Speaking  on  a  more  specific  level,  Sargent  suggests  that  there  are  three  basic 
decision-making  approaches  to  determining  model  validity  (Sargent,  1991:38).  In  the 
first  method,  the  modeler  or  model  development  team  makes  the  deciiiion  based  on 
various  tests  or  evaluations  conducted  with  the  model.  This  approach  is  the  most 
common.  It  is  usually  subjective  and  introduces  the  potential  for  modeler  bias.  The 
second  approach  uses  a  third  party  to  conduct  independent  verification  and  validation 
(IV&V)  of  the  model.  This  approach  reduces  the  chance  for  bias,  but  is  stiil  considered 
subjective.  IV&V's  greatest  disadvantage  is  its  cost. 

The  thirl  approach  involves  the  use  of  a  scoring  model.  Scores  on  various  aspects 
of  the  model  considered  important  to  the  model's  validity  are  assigned  by  the  modeler. 
These  scores  are  then  weighted  according  to  predetermined  factors  and  a  total  score  is 
calculated.  The  model  is  considered  valid  if  its  overall  and  category  scores  exceed  some 
determined  passing  score.  In  practice,  this  approach  is  seldom  used.  While  it  appears  to 
be  objective,  much  subjectivity  still  exists  in  the  assigning  of  category  scores. 
Furthermore,  category  weights  and  the  passing  score  are  difficuit  to  determine  in  practice 
(Sargent,  1991:38). 

In  all  of  the  above  validation  approaches,  various  techniques  may  be  used  to 
assess  model  validity.  Many  of  these  techniques  may  be  conducted  either  subjectively  or 
objectively  (using  a  statistical  test  or  procedure).  Usually,  a  combination  of  several 
techniques  is  used.  Some  of  the  mure  common  techniques  mentioned  by  Sargent  for 
determining  model  validity  are:  comparison  to  other  models,  extreme-condition  te.sts, 
face  validity,  historical  data  validity,  parameter  variability-sensitivity  analysis,  predictive 
validation,  and  Turing  tests  (Sargent,  1991:39-40). 

Another  concept  that  is  closely  related  to  the  idea  of  validity  is  credibility.  While 
model  validity  is  a  measure  of  how  well  the  model  represents  the  real  system,  credibility 
is  the  level  of  confidence  decision-ma.kRrs  have  based  on  model  information  they  have 
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received.  Demonstrating  high  model  validity  is  important  for  building  user  credibility, 
but  doesn’t  guarantee  the  model  will  be  deemed  credible  by  the  decision  maker.  Because 
credibility  actually  determines  whether  a  decision  maker  will  use  a  model,  it  is  important 
that  the  user  be  involved  as  much  as  possible  while  the  model  is  constructed.  This 
involvement  should  help  provide  the  additional  information  required  by  decision-makers 
to  establish  their  confidence  in  the  model.  If  the  modeler  is  able  to  develop  model 
credibility,  then  the  model  will  be  useful  and  the  decision-makers  should  be  willing  to  use 
the  model  (Sargent,  1991:37;  Shannon,  1975:208). 

Inference  —  Stage  5.  The  final  stage  of  Mihram's  model  building 
framework  is  the  inference  stage  (Stage  5  of  Figure  3-4).  This  stage  involves  using  the 
model  to  generate  information  for  use  by  the  decision  maker  or  modeler.  Experiments 
conducted  at  this  stage  are  done  with  the  model  exclusively.  Mihram  adds  that  most 
analytical  goals  at  tliis  stage  can  be  placed  into  three  general  categories:  1)  determination 
of  the  model's  dynamic  behavior  during  a  specific  time  period;  2)  determination  of  the 
marginal  effect  of  unit  changes  in  the  independent  variables  on  the  model's  expected 
responi.e  (dependent  variable)  at  the  end  of  a  specific  time  period;  and  3)  determination  of 
the  specific  value  of  the  independent  variables  which  maximizes  or  minimizes  the 
dependent  variable. 

Analvsis/Conclusion  -  Models  and  Modeling.  Our  literature  review  on 
models  and  modeling  supports  our  use  of  models  in  this  research  and  establishes  a  basis 
for  our  research  methodologies.  Through  the  literature  review,  we  were  able  to  define 
two  important  classes  of  models  used  in  problem  solving-conceptual  and  computerized 
models.  These  models  can  be  a  tool  for  decision  makers  to  help  analyze,  describe,  or 
predict  system  behavior  through  adherence  to  a  model-based  problem  solving  process. 
Shannon  describes  this  model-based  solving  process  as  an  iterative  method  where 
modelers  use  simulation  or  analyze  the  model  to  obtain  information  and  compare  the 
effects  of  various  decisions  or  policies  on  the  system. 
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Our  literature  review  on  modeling  convinced  us  that  modeling  was  an  appropriate 
tool  for  our  problem  and  provided  a  framewoilc  for  building  models.  While  there  is  much 
art  to  building  models,  there  is  significant  research  to  suggest  that  the  five  steps  proposed 
by  Mihram  can  help  increase  the  chance  that  the  model  is  useful  for  its  purpose.  This 
research  also  points  out  the  importance  of  clearly  identifying  the  problem  entity,  the 
model's  purpose,  and  the  management  issue;  all  three  of  which  are  necessary  to  build  and 
validate  the  model.  The  literature  review  was  also  helpful  in  identifying  numerous 
techniques  for  verifying  and  validating  models-important  steps  to  establishing  credibility 
with  the  model  user  or  decision  maker.  Finally,  in  the  inference  stage  the  modeler  draws 
conclusions  from  his  results  and  completes  the  model-based  problem  solving  process.  In 
conclusion,  these  findings  justify  our  effort  to  develop  a  conceptual  model  of  the  ACMP 
process  and  provide  a  valid  approach  to  conducting  our  research. 

First  Objective  —  Define  ACMP  Process  bv  Developing  a  Conceptual  Model 

This  thesis's  first  objective  was  to  develop  a  model  of  a  generic  ACMP  process. 
We  began  by  making  the  assumption  that  a  decision  maker  is  interested  in  solving  an 
existing  problem  with  a  mission  planning  process  and/or  improving  the  process.  Thus, 
the  decision  maker  is  faced  with  a  problem  situation. 

Our  literature  review  indicates  tliat  a  decision  maker,  when  faced  with  trying  to 
identify  a  specific  problem  or  solve  one,  is  often  aided  by  a  model  of  the  system  or 
process.  While  a  model  can  sometimes  give  answers  or  aid  in  making  choices,  it  almost 
always  aids  the  modeler  or  decision  maker  by  increasing  understanding  of  the  process  or 
system  (Hoeber,  1981:4-5).  Therefore,  the  development  of  a  model  for  this  thesis 
research  effort  made  sense  from  two  perspectives.  It  could  increase  our  knowledge  of  the 
mission  planning  process,  and  be  used  as  a  starting  point  for  the  creation  of  more  specific 
ACMP  models  to  be  used  by  other  decision  makers  or  analysts  in  evaluating  other 


3-14 


mission  planning  processes.  Having  justified  the  need  for  a  model,  we  set  out  to  build  a 
conceptual  model  of  the  ACMP  process. 

Building  the  Conceptual  Model.  To  build  the  conceptual  model  for  this 
research  effort  we  followed  a  framework  similar  to  that  proposed  by  Mihram  (Figure  3- 
4).  This  approach  involves  a  preliminary  goal  defining  step  (step  0),  followed  by  five 
model  building  steps:  1)  systems  analysis,  2)  system  synthesis,  3)  model  verification,  4) 
model  validation,  and  5)  inference  (Mihram,  1972:628).  While  this  approach  appears  to 
be  a  straight-through  sequential  process,  many  iterations  of  steps  0,1, 2,  and  3  were 
actually  performed  prior  to  attempting  model  validation. 

Modeling  Goals.  The  first  step  in  building  the  conceptual  model  was 
establishing  our  modeling  goals.  We  defined  the  problem  entity,  possible  measures  of 
effectiveness,  and  the  model's  purpose.  To  complete  these  steps,  we  compiled  and 
analyzed  information  obtained  during  the  literature  review  and  informal  discussions  with 
our  research  sponsors  at  HQ  TAG.  Our  personal  experience  aided  the  analysis.  From 
this  analysis,  drafts  of  the  problem  entity  definition  and  scope,  measures  of  performance, 
and  purposes  of  the  model  were  generated. 

In  trying  to  define  these  key  aspects  of  the  model,  we  found  it  difficult  to  avoid 
discussing  many  of  the  issues  Mihram  suggests  should  be  addressed  in  the  first  stage. 
System's  Analysis.  Therefore,  a  large  degree  of  overlap  between  the  model's  goal 
defining  stage  and  the  early  part  of  systems  analysis  stage  actually  occurred. 

Systems  Analysis.  Before  finalizing  the  problem  entity  and  the  model’s 
purpose,  systems  analysis  of  mission  planning  was  initiated.  The  systems  analysis  effort 
included  a  literature  review  on  topics  related  to  mission  planning  as  well  as  a  data- 
gathering  trip  to  collect  information  on  mission  planning  from  experts  at  HQ  TAG  and 
ESD.  This  stage  was  conducted  to  familiaiize  the  researchers  with  the  actual  mission 
planning  process  and  helped  to  define  the  goals  of  the  model.  Information  concerning 
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evems  and  functions  conducted  during  the  process,  the  timing  of  events,  key  individuals 
participating  in  the  process,  and  lessons  documented  from  Desert  Storm  were  of  primary' 
interest  during  the  analysis  phase. 

System  Synthesis,  At  about  midway  through  the  system  analysis  stage, 
discussions  about  how  the  conceptual  model  should  be  strucUired  and  graphically 
depicted  arose.  Therefore,  synthesis  of  the  conceptual  model  actually  began  before  the 
end  of  the  systems  analysis  stage,  and  w'as  conducted  concurrently  until  the  draft  model 
was  ready  for  expert  comments. 

Brainstorming  sessions  between  the  research  partners  were  held  several  times  a 
week  during  the  analysis/synthesis  stage  to  generate  ideas  for  model  illustration  and 
structure.  Flowcharts  of  mission  planning  process  events  and  data  sheets  for  each  process 
function  were  drafted  for  use  as  discussion  tools  and  to  test  our  understanding  of  the 
process.  As  these  flowcharts  and  data  sheets  matured,  attempts  to  integrr'e  further  model 
detail  such  as  planning  function  objectives,  data  requirements,  and  decisions  into  the 
model  were  attempted. 

Frequent  retreats  to  the  system  analysis  stage  were  made  as  questions  about  how 
the  process  functioned  were  raised  during  these  brainstorming  sessions.  Clarifications  on 
issues,  or  answers  to  questions  that  could  not  be  resolved  through  research  of  written 
literature  were  solicited  by  telephone  from  mission  planning  experts  at  HQ  TAC.  We 
obtained  second  sources  of  verificatiou  for  most  issues  not  specifically  documented  in  the 
written  literature. 

Verification.  As  the  structure  of  the  conceptual  model  began  to  take 
shape  and  the  number  of  modeling  changes  slowed,  a  single  baseline  version  of  the  model 
documentation  was  integrated.  Verification  of  the  model  was  conducted  against  this 
documentation  package.  A  foimai  draft  of  the  model  narrative,  graphics,  data  sheets  and 
backup  material  was  produced  from  the  baseline  and  checked  by  each  member  of  the 
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research  team  for  accuracy  and  clarity.  After  corrections  and  clarifications  were  made,  a 
final  copy  of  the  model  was  prepared  for  validation. 

Data  Collection  Methods.  During  the  analysis  stage  of  the  model  building 
process,  an  extensive  literature  review  was  conducted  to  familiarize  the  researchers  with 
academic  research  and  op  irrational  information  available  in  the  topic  areas  of  mission 
planning,  decision-making,  decision  support,  planning,  and  effectiveness/evaluation. 
In  addition,  information  on  lessons  learned  from  Desert  Storm  was  collected  and 
compiled.  Much  of  the  information  contained  in  Chapter  II  of  this  thesis  is  a  direct  result 
of  that  analysis  stage. 

A  visit  to  our  research  sponsors  at  HQ  TAC  and  ESD  afforded  us  the  opportunity 
to  discuss  various  aspects  of  the  mission  planning  process.  While  at  HQ  TAC,  we  spent 
two  and  a  half  days  discussing  unit-level  mission  planning  with  experts  at  the 
headquarters  and  collecting  data  for  our  thesis.  We  then  travelled  to  ESD  and  met  with 
the  SPO  director  for  the  new  automated  mission  planning  system-the  Air  Force  Mission 
Support  System  (AFMSS).  During  these  meetings  we  discussed  the  model's  purpose, 
scope,  and  level  of  detail  with  our  sponsors,  and  collected  feedback  on  these  issues  which 
we  incorporated  in  the  final  model.  We  believe  this  early  interaction  between  ourselves 
and  the  users  of  the  model  (sponsors)  enhanced  the  model's  credibility  and  usefulness. 

We  were  able  to  obtain  several  documents  that  helped  fill  in  missing  details  about  the 
mission  planning  process,  and  to  discuss  with  headquarters  and  development  personnel 
plans  for  the  future  of  mission  planning. 

Evidence  of  Data  Validity.  Both  academic  and  government  sources  of  data 
on  mission  planning  were  sought.  This  data-gathering  involved  collection  of  documents 
from  our  research  sponsors  as  well  as  published  reports  such  as  theses,  journal  and 
magazine  articles,  and  government  reports.  Due  to  the  lack  of  published  research  on  our 
thesis  topic,  many  of  the  details  on  the  mission  planning  process  were  obtained  from  Air 
Force  documents  or  discussions  with  mission  planning  experts  at  HQ  TAC. 
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An  assessment  of  data  validity  shoud  be  based  on  v/hether  the  data  was 
appropriate,  accurate,  and  sufficient  (Sargent,  1991:40).  In  terms  of  overall  data 
appropriateness,  all  of  the  information  collected  to  build  the  model  came  from  U.S. 
military  sources  or  from  academic  sources  writing  on  subjects  directly  related  to  U.S.  Air 
Force  mission  planning.  Because  the  ACMP  process  is  unique  to  the  military  and  the 
model  was  developed  to  be  generic  or  conceptual,  these  sources  are  considered 
appropriate. 

During  systems  analysis,  a  preliminary  check  on  the  accuracy  of  our  data  was 
made  during  our  data  collection  visit  with  our  sponsors.  Model  information  in  draft  form 
was  presented  to  the  sponsors  who  offered  comments  and  corrections  on  the  draft  model. 
Internal  consistency  checks  were  conducted  by  the  researchers  to  help  verify  the  model 
data.  A  database  of  Journal  articles,  books,  and  reports  was  maintained  by  the  researchers 
to  organize  the  information  for  report  preparation  and  to  prevent  data  loss. 

Model  Vaiidation.  After  completing  the  analysis,  synthesis  and  verification 
steps  of  the  modeling  process,  we  attempted  to  validate  the  conceptual  model  with 
respect  to  each  stated  purpose.  This  validation  approach  is  consistent  with  the 
recommendation  proposed  by  Sargent  that  model  validity  should  be  determined  with 
regard  to  each  puqTose  of  the  model: 

Conceptual  model  validity  is  determining  that  the  theories  and  assumptions 
underiying  the  conceptual  model  are  correct,  and  that  the  model  representation 
of  the  problem  entity  and  the  model's  structure,  logic,  and  mathematical  and 
causal  relationships  are  'reasonable'  for  the  intended  purpose  of  the  model. 

(Sargent,  1991:40) 

As  suggested  by  Sargent,  a  primary  validation  technique  used  for  conceptual  model 
validation  is  face  validation  (Sargent,  1991:41).  Face  validation  involves  having  experts 
evaluate  and  comment  on  the  overall  conceptual  model  to  determine  if  it  is  correct  and 
reasonable  for  its  intended  purpose.  Face  validation  usually  involves  examination  of 
flowcharts,  graphics,  and  model  information  to  determine  if  the  model  logic  is  correct  and 
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the  model's  accuracy  is  maintained  (Sargent,  1991;^  1).  We  used  this  approach  to  validate 
our  conceptual  model. 

We  selected  a  convenience  sample  of  five  mission  planning  experts  from  the 
mission  planning  program  offices  at  HQ  TAG  and  ESD,  based  on  their  mission  planning 
experience,  demonstrated  knowledge,  interest,  and  availability.  We  then  contacted  them 
and  asked  if  they  would  be  willing  to  comment  on  our  model.  Of  the  five  individuals 
selected,  four  agreed  to  participate.  Because  the  fifth  expert  could  not  be  reached  prior  to 
sending  out  the  model  package,  a  replacement  expert  was  asked  to  participate.  The 
replacement  expert’s  participation  gave  us  a  total  of  five  mission  planning  experts. 

A  moriel  validation  package  was  mailed  to  each  of  the  five  mission  planning 
experts  that  agreed  to  participate.  Each  package  consisted  of  the  following  items: 

1)  A  cover  letter  explaining  the  purpose  of  the  research  and  how  the  validation  is  being 
conducted. 

2)  A  comment  sheet  requesting  demographic  information  about  the  expert  and 
containing  questions  to  be  answered  by  the  expert  about  various  aspects  related  to  the 
model's  validity. 

3)  A  narrative  introduction  to  the  model  which  addressed  the  model’s  purpose,  scope, 
level  of  detail,  model  boundaries,  assumptions,  and  data  categories. 

4)  The  conceptual  model  portrayed  in  graphical  format. 

5)  Detailed  narrative  information  as  backup  to  the  model  (function  descriptions,  data 
examples,  and  function  summary  sheets. 

The  experts  were  asked  to  review  the  package,  complete  the  comment  sheets,  and  return 
them  in  the  enclosed  envelope.  Upon  receipt  of  their  completed  comment  sheets,  we 
contacted  each  of  the  experts  by  telephone  to  follow-up  on  their  comments  and  discuss 
changes  to  the  model  that  might  correct  its  flaws.  If  possible  and  acceptable,  the  changes 
were  then  incorporated  into  the  model.  Due  to  the  time  constraints  of  this  research 
project,  no  follow-up  validation  packages  were  sent  to  the  experts.  The  results  of  this 
model  building  objective  and  the  validation  assessment  are  addressed  in  the  results 
section  of  this  thesis  (Chapter  IV). 
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Second  Objective  —  Evaluating  ACMP  Process  Effectiveness 

Our  second  objective  was  to  derive  from  previous  research  a  general  method  for 
evaluating  the  effectiveness  of  an  ACMP  process.  This  objective  was  primarily 
accomplished  through  a  second  phase  of  literature  review  of  research  related  to  system  or 
process  effectiveness  evaluations.  Our  preliminary  literature  review  suggested  that 
methodologies  for  measuring  elements  of  the  squadron  ACMP  process  can  be  found  in 
many  academic  areas.  Command  and  conUt>l  systems,  and  decision  support  systems, 
combined  with  techniques  from  the  social  sciences,  offer  methods  of  evaluating  the 
ACMP  process  at  a  level  where  we  can  evaluate  component  contribution  to 
effectiveness.  Planning,  decision-making,  and  total  quality  process  improvement  are 
areas  of  specialization  we  identified  that  offered  the  potential  of  overall  ACMP  process 
effectiveness  measures. 

Several  authors  have  suggested  that  the  appropriateness  of  evaluation  methods 
depends  on  the  evaluation  purpose  (Sprague  and  Carlson,  1982:167;  Reidel  and  Pitz, 
1986:994;  Saaty,  1990:4-6).  Additionally,  researchers  have  concluded  that  given  the 
various  strengths  and  weaknesses  of  different  methods,  combinations  of  methods  often 
provide  the  best  results  (Sprague  and  Carlson,  1982:167;  Reidel  and  Pitz,  1986:989).  We 
have  primarily  focused  on  identifying  methods  that  support  the  evaluation  purpose  of 
operational  command  appraisal  of  overall  ACMP  process  elTcctiveness.  Additionally, 
we  have  attempted  to  integrate  into  the  methodology  measures  that  differentiate 
component  contribution  to  effectiveness.  However,  this  additional  benefit  was  weighed 
against  the  additional  complexity  forced  on  the  field  evaluators.  Our  goal  was  to  identify 
methods  that  could  be  used  by  field  personnel  within  the  framework  of  the  conceptual 
model  developed  in  the  first  objective.  Of  the  methods  identified,  we  distinguished  the 
parameters  used  to  evaluate  process  effectiveness.  The  degree  of  compatibility  between 
the  model  and  method  parameters  is  an  important  criterion  for  method  selection.  After 
evaluating  published  approaches,  we  derived  a  proposed  evaluation  methodology.  The 
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proposed  evaluation  methodology  was  validated  based  on  its  derivation  from  previous 
work,  and  feedback  from  proposed  users  and  experts.  Additional  validation  would  come 
from  work  on  the  third  objective,  demonstration  of  the  approach. 

Third  Objective  -  Demonstrate  Application  of  ACMP  Evaluation  Method 

The  third  objective  of  our  thesis  was  to  demonstrate  the  application  of  the 
conceptual  ACMP  model  resulting  from  research  objective  #1,  and  the  general  ACMP 
process  evaluation  method  developed  in  research  objective  #2,  to  the  investigation  of  a 
specific  ACMP  management  issue.  This  objective's  primary  goal  was  to  test  the 
reasonableness  of  our  developed  model  and  evaluation  method  for  use  by  operational 
managers  or  decision  makers.  A  secondary  goal  of  this  objective  was  to  provide  an 
example  of  our  proposed  evaluation  methodology  to  clarify  how  it  might  be  conducted  in 
an  actual  situation.  We  first  discuss  our  approach  to  the  demonstration  and  then  the 
specific  demonstration  scenario  we  developed. 

Approach.  The  specific  management  issue  we  selected  for  this  demonstration 
was  one  of  several  suggested  by  HQ  ACC.  Aircraft  system  developers  have  asked  HQ 
ACC  what  amount  of  time  is  available  in  the  flight  planning  process  for  performing  new 
planning/preparation  functions.  Our  specific  management  issue  was  to  develop  an 
answer  for  HQ  ACC.  We  chose  the  problem  based  on  its  perceived  moderate  degree  of 
complexity  and  its  suitability  for  illustrating  our  evaluation  approach.  The  problem  is 
part  of  a  larger  issue  mission  planning  managers  are  currently  facing  in  the  operational 
community-determining  the  effects  of  automated  mission  planning  systems  on  flight 
planning  time  and  plan  quality  in  the  squadron.  It  is  important  to  point  out  that  this 
application  of  the  ACMP  approach  is  a  demonstration.  Therefore,  we  simplified  the 
problem  for  purpose  of  illustration.  Simplifications  were  also  made  to  the  model  as  a 
result  of  our  research  time  constraints.  We  believe  these  simplifications  are  realistic 
because  operational  users  may  have  similar  constraints. 
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The  method  we  used  to  accomplish  this  objective  was  the  ACMP  process 
evaluation  method  we  developed  in  objective  #2.  We  first  defined  the  management  issue, 
selected  our  measures  of  plan  quality  from  the  two  vectors  suggested  by  Metersky, 
determined  our  treatment  levels  or  dependent  variables,  and  then  developed  our  specific 
models  of  the  process  (our  alternatives).  With  these  alternative  models,  we  conducted 
simulation  experiments  to  generate  information  that  could  assist  us  in  proposing  a 
management  solution  or  evaluation  result. 

For  our  simulation  experiments,  we  elected  to  build  and  conduct  a  computer 
simulation  using  computerized  models  of  the  ACMP  process.  The  starting  point  for  these 
alternative  models  was  the  conceptual  model  built  in  research  objective  #1.  From  that 
conceptual  model,  and  using  the  information  contained  in  the  problem  scenario,  the 
computerized  models  of  the  ACMP  process  were  developed  using  Mihram's  model 
building  approach.  The  actual  building  and  simulation  of  the  computerized  models  was 
accomplished  using  FORTRAN  subroutines  and  a  specialized  simulation  and  modeling 
language  developed  by  Pritsker  &  Associates  called  Simulation  Language  for  Alternative 
Modeling  11  (SLAM  II).  SLAM  II  was  chosen  for  its  flexibility  (permits  network, 
discrete  event,  continuous,  and  combined  computer  simulation  modeling  approaches), 
availability  (available  to  AFIT  students)  and  familiarity  to  the  researchers.  While  we 
chose  to  use  SLAM  11  for  this  demonstration,  our  evaluation  methodology  does  not 
require  its  use.  The  only  requirement  of  our  approach  is  that  the  modeler  or  decision 
maker  develop  ACMP  process  models  and  conduct  simulation  or  experiments  to  evaluate 
the  models. 

After  conducting  the  computer  simulation,  we  analyzed  the  results  in  order  to 
provide  information  concerning  the  mission  vector  aspects  of  plan  quality  for  each  model 
alternative.  From  the  results  of  the  mission  vector  analysis,  a  proposed  management 
solution  was  chosen  from  the  alternatives  for  further  consideration  by  the  decision  maker. 
Before  a  final  decision  could  be  made,  the  behavioral  factors  of  plan  quality  not 
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operationalized  in  the  computerized  model  needed  to  be  considered.  As  our  literature 
review  points  out,  there  is  no  formal  way  to  conduct  this  evaluation  of  behavioral  factors. 
Therefore  we  assessed  the  effects  of  any  behavioral  factors  as  we  expect  method  users 
might-through  personal  or  expert  opinion,  backed  up  by  any  published  research  if 
available.  We  selected  behavioral  factors  that  we  believed  might  affect  the  performance 
of  the  process  and  then  attempted  to  determine  the  direction  and  amount  of  change.  After 
considering  the  effects  of  both  mission  and  behavioral  determinants  of  plan  quality  on  the 
proposed  solution  to  the  problem,  we  finalized  our  results. 

Description  of  the  Problem  Scenario.  To  support  our  demonstration  of  the 
ACMP  Evaluation  Methodology  we  worked  with  HQ  ACC  to  develop  the  following 
problem  scenario. 

System  developers  are  adding  a  new  capability  to  existing  fighter  aircraft.  This 
new  capability  will  require  some  planning  and  preparation  by  the  aircrew  on  the  mission 
planning  computers  during  the  flight  planning  process.  System  developers  have  asked 
HQ  ACC  what  amount  of  time  is  available  in  the  flight  planning  process  for  performing 
this  new  planning/preparation  function.  To  answer  the  question,  HQ  ACC  has  decided  to 
answer  the  question  using  the  evaluation  methodology  developed  in  our  thesis. 

HQ  ACC  wants  to  evaluate  the  situation  in  a  near  worst-case  flight  planning 
scenario  with  respect  to  time,  in  which  a  squadron,  notionally  an  F-16  unit,  is  tasked  to 
conduct  six  four-ship  missions  under  combat  surge  surge  operations.  These  missions  are 
90  minutes  in  duration.  These  short  mission  durations  allow  each  crew  member  to  plan 
and  fly  two  missions  within  a  12-hour  crew  day. 

Each  day  the  ATO  is  received  at  the  wing  for  the  next  day's  tasking.  Mission 
Commanders  and  a  mission  planning  cell  prepare  package  plans  and  preliminary  flight 
planning  material  from  the  time  the  ATO  is  received  until  the  beginning  of  mass  brief. 
Mass  brief  is  alv/ays  scheduled  3  hours  before  the  first  flight's  take-off  time.  After 
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attending  a  mass  brief,  that  lasts  30  minutes,  each  flight  can  start  flight  planning  activities 
if  sufficient  mission  planning  stations  are  available. 

Bight  planning  at  the  squadron  is  accomplished  with  the  aid  of  mission  planning 
computers.  The  standard  number  of  mission  planning  stations  in  an  F-16  squadron  is 
three.  Within  a  four-ship  flight,  all  four  pilots  are  assumed  to  participate  in  mission 
planning.  But,  only  up  to  two  of  the  pilots  from  a  particular  flight  will  use  mission 
planning  stations.  The  pilots  not  using  stations  may  perform  support  tasks  such  as  data 
gathering.  Bight  planners  would  prefer  to  use  two  mission  planning  stations  to  build 
their  flight  plan.  If  two  mission  planning  systems  are  available,  the  flight  planners  will 
use  them.  However,  if  only  one  station  is  available,  the  flight  will  stan  planning  on  the 
single  station  until  the  second  station  becomes  available.  Priority  for  station  use  is 
determined  by  take-off  time.  The  flight  with  the  earliest  take-off  time  has  priority. 

Bights  already  performing  flight  planning  have  priority  over  those  who  have  not  begun. 

HQ  ACC  has  determined  that  flight  planning,  using  two  mission  planning  stations 
for  a  single  flight,  currently  requires  10  to  20  minutes  of  planning  time.  Of  that  planning 
time,  about  three-fourths  of  it  is  spent  building  the  plan  .  The  remaining  one-fourth  of  the 
time  is  spent  integrating  the  different  pieces  and  preparing  the  mission  materials.  When 
only  one  mission  planning  station  is  available,  the  time  needed  to  build  the  initial  parts  of 
the  plan  is  doubled.  After  initial  flight  planning  by  both  pilots,  one  of  the  pilots  must 
remain  on  a  single  station  to  integrate  the  flight  plan  and  prepare  the  mission  material. 
Once  a  flight  begins  flight  plan  integration,  it  must  release  its  second  mission  planning 
station  if  two  are  being  used. 

After  completing  flight  planning  on  the  computers,  any  time  available  until  the 
crew's  step  time  (slack)  can  be  used  by  the  ~rews  to  study  their  mission  plan.  However, 
each  flight  must  conduct  a  flight  briefing  followed  by  ground  operations  before  take-off. 
The  time  needed  to  conduct  the  flight  briefing  and  ground  operations  varies  uniformly 
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between  60  -  80  minutes.  Six  four-ship  flights  are  tasked  in  each  package.  The  first  take¬ 
off  time  is  scheduled  three  hours  after  the  beginning  of  the  mass  brief.  After  the  first 
flight  takes-off,  remaining  flights  take  off  at  10  minute  intervals.  Take-off  times  cannot 
be  missed  or  slipped.  If  time  is  not  available  to  complete  the  flight  plan,  the  crews 
proceed  with  an  incomplete  plan  and  must  still  fly  the  mission.  Mass  briefs,  flight 
briefings  and  all  ground  operations  must  occur  and  their  times  cannot  be  reduced.  The 
number  of  incomplete  flight  planning  sessions,  the  amount  of  lost  planning  time,  and 
slack  time  is  of  interest  to  management. 

Each  pilot  flies  two  90  minute  (from  take-off  to  landing)  missions  a  day  within  a 
12  hour  period  (a  crew  day).  A  crew  day  begins  when  the  crew  member  shows  up  for  the 
mass  brief  and  ends  after  the  last  post-flight  debriefing.  The  crew  day  can't  be  extended 
due  to  policies  on  crew  rest.  After  completing  an  initial  mission,  crews  must  perform 
post-flight  ground  operations  and  debriefing  before  picking  up  a  second  package  of 
preliminary  flight  materia]  and  initiating  flight  planning  for  a  second  mission  (no  second 
mass  brief).  The  time  required  to  complete  post  mission  ground  operations,  debrief,  and 
pick  up  a  new  mission  package  varies  uniformly  between  80  - 100  minutes.  Take-off 
times  for  the  second  set  of  sorties  begin  5  hours  and  25  minutes  after  the  first  take-off 
time  of  the  day. 

To  increase  the  chances  for  a  quality  mission  plan,  HQ  ACC  wants  to  make  sure 
that  aircrews  have  sufficient  time  on  the  mission  planning  stations.  So,  we  want  to  avoid 
having  crews  abort  their  mission  planning  sessions  early  to  make  their  step  times  or  take¬ 
off  times  because  plan  quality  would  become  unacceptable.  To  answer  the  developer's 
question,  HQ  ACC  wants  to  determine  how  much  time  can  be  added  to  a  flight's  planning 
time  before  plan  quality  begins  to  suffer.  Our  response  to  the  developer’.s  question  is 
provided  in  Chapter  IV 's  Objective  #3  results. 
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Summary 

Our  thesis  research  had  three  related  research  objectives.  We  developed  a  model- 
based  problem  solving  meta-methodology  based  on  our  preliminary  analysis  and 
subsequent  literature  review.  The  model-based  problem  solving  process,  like  other 
scientific  approaches,  has  been  successfully  used  by  many  other  researchers  which  gave 
us  confidence  in  the  approach.  As  previously  indicated,  this  approach  seemed 
particularly  useful  from  two  perspectives.  Developing  the  ACMP  model  documents  an 
important  process  that  the  Air  Force  seeks  to  improve  and  it  serves  as  a  starting  point  for 
evaluation.  For  each  objective  we  developed  an  appropriate  methodology  for  completing 
our  research.  Our  use  of  each  methodology  is  justified  by  its  derivation  from  previous 
research  and  adherence  to  model-based  problem  solving  approach  principles.  While  use 
of  this  approach  did  not  guarantee  success,  our  consistency  with  prior  research 
indicated  our  meta-methodology  was  a  reasonable  approach  for  proceeding  with  thesis 
research. 
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IV.  Results 


Overview 

The  results  from  our  research  are  presented  in  the  following  sections  on  each  of 
our  related  three  objectives:  conceptual  model,  proposed  ACMP  evaluation  methodology, 
and  demonstration  of  the  proposed  methodology.  The  results  from  each  objective  are 
presented  separately.  We  present  conclusions  from  our  integrated  results  in  Chapter  V. 
The  conceptual  model  includes  a  description  of  both  the  ACMP  process  and  graphical 
depictions  of  the  process.  The  model  is  supported  by  detailed  material  in  the  appendices. 
The  section  on  our  proposed  ACMP  evaluation  methodology  explains  our  derivation  of 
the  methodology  and  the  proposed  process  for  evaluation.  We  conclude  the  results  with  a 
discussion  of  our  demonstration  of  the  ACMP  evaluation  methodology. 

Objective  #1  —  Conceptual  Model 

The  first  objective  of  our  research  on  the  Air  Combat  Mission  Planning  (ACMP) 
process  was  to  develop  a  conceptual  model  of  the  ACMP  process.  A  conceptual  model  is 
defined  as  a  logical,  mathematical  or  verbal  representation  of  a  problem  entity  that  is 
developed  for  a  particular  study  (Sargent,  1991:38).  In  our  selected  model  building 
approach,  the  first  stage  of  model  building  required  us  to  define  the  problem  entity.  For 
this  research,  the  problem  entity  (the  system,  idea,  situation,  policy,  or  phenomena  to  be 
modeled)  is  the  ACMP  process.  The  second  stage  of  the  model  building  process 
required  us  to  collect  information  and  analyze  the  system.  The  information  used  to 
develop  this  model  c  ime  from  various  sources.  They  Included  a  literature  review  of 
related  mission  planning  research,  discussions  with  Air  Force  mission  planning  experts, 
and  the  personal  experience  of  the  researchers. 

As  we  proceeded  through  the  model  building  process,  we  refined  the  model’s 
purpose,  scope,  level-of-detail,  boundaries,  assumptions,  and  data  categories.  These  are 
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presented  in  the  following  sections.  Graphical  depictions  of  the  model's  elements  and 
relationships  are  then  presented. 

Definitions.  The  following  definitions  are  used  throughout  the  conceptual 


model: 

ATO  —  Air  Tasking  Order.  The  mechanism  used  to  provide  daily  tasking  to  all  units 
supporting  the  theater  air  campaign. 

Ftight  -  The  lowest  level  mission  element.  Flights  are  usually  made  up  of  two  or  four 
aircraft 

FUght  Leader  —  The  individual  responsible  for  the  planning,  coordination,  and  execution 
of  the  flight's  portion  of  the  mission. 

Frag  »  Fragmentary  Order.  A  subset  of  the  ATO  that  is  sent  to  subordinate  squadrons. 
The  Frag  has  been  reviewed  and  processed  at  the  wing  to  assign  resources  (aircraft  and 
aircrews). 

Mission  Commander  -  The  individual  responsible  for  package  planning,  coordination, 
and  execution  of  the  mission.  This  individual  is  typically  an  experienced  pilot  selected 
from  the  unit  that  is  providing  the  primary  mission  aircraft.  The  ATO  specifies  who  or 
what  unit  will  provide  the  mission  commander  for  each  package. 

Package  -  The  largest  mission  element.  A  package  can  consist  of  many  different 
aircraft  from  several  wings  and  squadrons  performing  in  strike/attack  and  support  roles 
during  a  mission. 

Sortie  -•  One  flight  by  a  single  aircraft. 

Purpose  of  the  Model.  The  purpose  of  this  conceptual  model  is  two-fold. 

First,  it  establishes  a  framework  of  mission  planning  functions,  decisions  and  data 
categories  that  when  combined,  provide  a  process  description  and  mission  planning 
taxonomy  for  those  new  to  the  concept  of  ACMP.  Second,  the  model  establishes  a  base 
for  the  creation  of  more  specific  ACMP  models  that  are  necessary  for  evaluating  newly 
proposed  or  current  ACMP  processes. 


Scope  of  the  Process.  For  this  model,  the  ACMP  process  includes  the 
functions,  decisions,  and  activities  that  occur  in  an  operational  wing  and  squadron  during 
wartime.  These  functions,  decisions  and  activities  transform  the  unit's  tasking  found  in 
the  Air  Tasking  Order  (ATO)  into  individual  flight  plans,  data,  and  materials  used  by  the 
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individual  aircrews.  Also  included  in  this  ntodel  are  the  activities  associated  with  the 
loading  of  the  mission  data  into  the  aircraft.  With  the  exception  of  the  cases  mentioned 
below,  variations  in  the  mission  planning  process  that  may  occur  from  one  squadron  or 
wing  to  the  next,  should  fit  within  the  framework  established  by  this  model. 

Training  missions,  the  planning  of  Single  integrated  Operational  Finn  (SIOP) 
nuclear  missions.  Special  Operations  Forces  (SOF)  missions,  and  suppoit  to  “black" 
programs  are  omitted  from  this  model  because  their  planning  processes  diverge  from  core 
requirements  and  their  inclusion  would  severely  complicate  the  model. 

Level  of  Detail.  The  amount  of  detail  included  in  any  model  should  depend 
upon  the  purpose  of  the  model  (Pritsker,  1991:1200).  Because  the  purpose  oi'this  model 
is  to  provide  a  base  for  either  understanding  the  process  or  building  more  specific  models, 
a  high  level-of-detail  was  traded  for  more  model  breadth  and  generality.  The  intent  was 
to  capture  the  major  elements,  decisions  and  dependencies,  not  to  inundate  the  reader 
with  minute  detail.  This  approach  fits  with  the  model's  goal  of  a  descriptive  framework 
but  should  not  limit  its  use  by  experts  as  a  starting  point  for  more  specific  models. 

Experts  can  benefit  by  starting  with  a  broad  perspective  thereby  reducing  the  chances  of 
leaving  out  important  details  and  elements  of  a  specific  model. 

Boundaries  of  the  Model.  The  boundaries  of  the  model  exist  at  the  point  in 
time  at  which  the  ATO  is  received  from  HHQ  by  the  wing  and  ends  at  the  point  in  time  at 
which  the  aircraft  is  initialized  for  this  mission.  Side  boundaries  exist  at  the  interface 
with  the  mission  planners  and  the  external  functions  that  impact  planning  execution,  but 
are  not  controlled  by  the  mission  planners.  These  boundaries  are  consistent  with  the 
ACMP  process  definition  that  includes  those  activities  which  occur  at  the  unit  level  (wing 
and  squadron)  from  the  time  the  Air  Tasking  Order  (ATO)  is  received  at  the  wing  until 
the  aircraft  is  loaded  with  all  of  the  mission  specific  information  prior  to  takeoff.  Only 
those  activities  related  to  the  actual  generation  of  the  mission  plan,  flight  plan,  or  crew 
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materials  by  the  mission  planners  are  included  in  the  model.  Activities  or  tasks  which 
provide  support  functions  such  as  intelligence  or  weather  analysis  and  directly  support 
the  mission  planners  are  excluded  because  they  exist  on  the  other  side  of  the  planner's 
inibrmation  interface.  Their  influence  on  the  process  is  modeled  as  an  information  or  data 
input,  and  are  in  a  systems  sense,  part  of  the  model’s  environment. 

Model  Assumptions.  We  made  four  important  simplifying  assumptions  for 
developing  the  conceptual  model. 

1.  For  purpose  of  the  model,  it  is  assumed  that  the  majority  of  package  planning, 
as  it  affects  the  daily  mission  planning  process,  occurs  after  receipt  of  official  tasking. 

2.  Tasking  is  received  in  the  Air  Tasking  Order  (ATO)  every  24  hours. 

Retasking,  either  as  a  change  to  an  existing  mission  or  the  addition  of  a  new  mission,  is 
considered  only  to  the  extent  that  it  appears  as  an  input  to  the  system.  No  attempts  to 
model  the  effect  of  retasking  on  the  general  mission  planning  functions  have  been  made. 

3.  The  model  is  presented  from  the  perspective  of  the  location  of  the  mission 
commander.  Flight  leaders  assume  the  responsibilities  of  mission  commander  at 
supporting  units  (e.g.,  tankers  and  electronic  warfare). 

4.  Information,  when  needed,  is  available. 

Categories  of  Mission  Piannin2  Data  used  in  the  Model.  To  be 
consistent,  the  conceptual  model  classifies  the  data  and  information  used  by  ACMP 
planners  into  five  categories.  These  categories  and  their  descriptions  were  adapted  from 
the  concept  of  operations  document  for  the  new  Air  Force  mission  planning  system  being 
developed  by  Air  Force  Material  Command  (HQTAC/DRIB,  1991:4-8). 

Parametric  Data- Parametric  data  defines  aircraft  and  weapons  performance  and 
signature  models.  It  is  used  within  algorithms  for  such  items  such  as  fuel  flow 
calculations,  radar  cross  section,  or  as  the  basis  for  threat  models. 
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MCG&l  Data-MCG«kI  data  includes  digital  maps,  charts,  terrain  elevation, 
cultural  features,  imagery,  and  other  types  of  aeronautical  information. 

Operational  Information-Ooerationat  information  is  input  data  relevant  to  the 
unit's  area  of  interest,  including  intelligence  data  (e.g.,  enemy  threats),  operations  data 
(e.g.,  ATO),  and  weather  data.  This  data  is  the  basis  for  planning  optimum  ingress  and 
egress  routes  and  mission  accomplishment. 

Mission  Infonnation --Mission  information  is  the  result  of  the  mission  planning 
process  and  the  mission  itself.  The  user  interacts  with  planning  systems,  algorithms,  and 
operational,  MCG&I,  and  parametric  databases,  to  produce  plans,  a  CMF  (combat 
mission  folder)  and  load  a  DTD  (Data  Transfer  Device).  The  DTD  contains 
aircraft/weapons  navigation  and  avionics  initialization  data. 

Setup  Information-Setup  information  includes  unit/user  unique  data  and  standard 
world-wide  navigation  points  and  information  (way  point  library).  The  unit/user  data 
includes  security  data,  and  profiles  for  default  settings,  mission  planning,  and  forms/CMF 
generation. 

Each  of  the  above  categories  was  broken  down  to  a  second  tier  to  provide  more 
model  detail.  Further  discussion  about  these  categories,  including  a  list  of  the  second  tier 
categories  and  examples  of  possible  third  tier  categories,  is  provided  within  the  detailed 
backup  material  at  Appendix  B. 

The  ACMP  Conceptual  Model.  The  ACMP  conceptual  model  was 
developed  and  validated  as  a  package  including  a  narrative  overview,  graphical 
depictions  of  the  modeled  ACMP  process,  and  a  data  requirements  matrix.  The  narrative 
overview  provided  the  model’s  purpose,  scope,  level-of-detail,  boundaries,  assumptions 
and  data  categories  as  previously  presented  in  this  chapter.  Within  the  context  of  the 
model,  the  ACMP  process  is  broken  down  into  three  phases  and  then  into  functions.  This 
breakdown  into  phases  and  functions  facilitated  discussion  and  understanding.  Each 
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function  is  further  defined  by  its  objective,  primary  decision(s),  level  2  data  categories, 
and  products.  The  three  modeled  phases  are  Before  Flight  Planning,  Flight  Planning, 
and  After  Flight  Planning.  Given  the  communicative  purposes  of  the  conceptual  model, 
graphic  depictions  are  used  to  convey  the  data  gathered  on  each  function. 

Figure  4-1  provides  an  overview  of  the  unit-level  ACMP  process  showing  the 
flow  of  information  between  functions  and  through  the  process.  The  heavy  dashed  line 
encompasses  the  functions  considered  as  part  of  the  ACMP  “system”  or  process.  Figure 
4-2  graphically  depicts  the  interfaces  between  the  process  and  its  external  environment 
and  illustrates  the  important  input  and  feedback  mechanisms  for  support  and  control. 
Figures  4-3  through  4-1 1  identify  the  function  objectives,  decisions,  data,  and  products  of 
the  process  and  their  relationships. 

Figures  4-3  through  4-5  provide  the  objectives  for  each  of  the  three  phases.  The 
Objectives  Before  Flight  Planning  (Figure  4-3)  support  the  need  to  breakout  the  force- 
level  tasking  and  prepare  for  flight  planning.  The  Flight  Planning  Objectives  (Figure  4- 
4)  represent  the  goals  crew  members  seek  to  accomplish  during  flight  planning.  These 
functions  are  not  sequentially  ordered  and  may  be  accomplished  in  any  order.  However, 
the  work  generally  starts  from  the  top  continuing  down  and  moves  from  left  to  right. 
Figure  4-5,  Objectives  After  Flight  Planning,  shows  the  need  to  integrate  the  work  of 
flight  planning  participants  and  prepare  for  mission  execution. 

Figures  4-6  through  4-8  identify  the  decisions  that  must  be  made  during  each  of 
the  three  ACMP  phases.  Decisions  Before  Flight  Planning,  Figure  4-6,  reflects  the 
assessment,  allocation,  and  preparatory  tasks  that  are  performed  during  this  phase.  The 
decisions  of  the  Flight  Pre-Planning  function  are  a  situationally  determined  subset  of 
Flight  Planning  phase  decisions.  For  example,  wings  may  use  Mission  Planning  Cells  to 
reduce  the  planning  that  must  be  accomplished  during  crew  duty  days.  Flight  Planning 
Decisions  (Figure  4-7)  range  from  relatively  automatic  Premission  TOLD  decisions  to 
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more  complex  Target  Area  Tactics  decisions.  These  flight  planning  decisions  reflect  the 
need  to  plan  for  operation  of  the  involved  aircraft  and  weapons,  to  plan  ingress  and  egress 
routes,  and  to  plan  combat  operations.  Relatively  few  planning  decisions  are  made  in  the 
After  Flight  Planning  phase  (Figure  4-8).  The  Ground  Operations  function  does  not  have 
any  flight  planning  decisions. 

Figures  4-9  through  4-1 1  present  data  requirements  on  the  fronts  of  the  function 
cubes  and  products  on  the  sides  of  the  function  cubes  for  each  of  the  three  ACMP  phases. 
While  only  the  top-level  information  is  presented,  additional  detail  is  available  in 
Appendix  B  and  Appendix  C.  Planners  require  increasingly  detailed  data  as  planning 
progresses  Before  Flight  Planning  (Figure  4-9).  The  primary  products  are  the  package 
plan  and  flight  planning  guidance.  Each  function  in  the  Flight  Planning  phase  (Rgure  4- 
10)  has  specific  data  requirements  and  produces  specific  products.  However,  interim  data 
and  products  from  other  Flight  Planning  functions  may  be  used  due  to  the  iterative  nature 
of  flight  planning.  Flight  Plan  integration  has  the  majority  of  data  requirements  and 
produces  most  of  the  products  from  the  After  Flight  Planning  phase  (Figure  4-11). 

Figure  4-12  A-B  provides  an  ACMP  mission  planning  data  requirements  matrix 
that  quickly  summarizes  the  information  requirements  and  data  dependencies  for  each 
function.  Detailed  backup  material  that  provides  additional  information  and  examples  are 
provided  at  Appendix  A,  B  and  C.  This  backup  information  is  not  formally  part  of  the 
model,  but  may  provide  some  insight  to  areas  that  are  less  familiar  to  the  reader. 
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Unit-Level  Air  Combat  Mission  Planning 
With  Interfaces 


4-1.  Unit-Level  ACMP  With  Inteifaces 


Unit-Level  Air  Combat  Mission  Planning 
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Figure  4-2.  Unit-Level  Air  Combat  Mission  Plai 
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Figure  4-3.  Objectives  Before  Flight  Planning 


Flight  Planning  Objectives 


Figure  4-4.  Flight  Planning  Objectives 


Objectives  After  Flight  Planning 


Figure  4-5.  Objectives  After  Flight  Planning 


Decisions  Before  Flight  Planning 


Figure  4-6.  Decisions  Before  Flight  Planning 


Flight  Planning  Decisions 


Figure  4-7.  Flight  Planning  Decisions 


Decisions  After  Flight  Planning 
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Figure  4-8.  Decisions  After  Flight  Planning 


Data  Required  and  Products  Before  Flight  Planning 


Figure  4  9.  Data  Required  and  Products  Before  Flight  Planning 


Flight  Planning  Data  Required  and  Products 
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Figure  4-10.  Flight  Planning  Data  Required  and  Products 


Data  Required  and  Products  After  Flight  Planning 
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Figure  4-11.  Data  Required  and  Products  After  Flight  Planning 


Air  uombat  Mission  Planning  Data  Requirements  and  Products 
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4-12A.  ACMP  Daia  Reqnheineiits  Matrix  (Sheet  L  of  2] 
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4-12B.  ACMP  Data  Reqtmemetils  Matrix  (Sheet  2  of  2) 


Conceptual  Model  Validation.  As  discussed  in  our  methodology  chapter,  the 
primary  technique  used  to  validate  our  conceptual  model  was  face  validation.  Face 
validation  involves  having  experts  evaluate  and  comment  on  the  overall  conceptual 
model  to  determine  if  it  is  correct  and  reasonable  for  its  intended  purpose.  Face 
validation  usually  Involves  examination  of  flowcharts,  graphics,  and  model  information 
to  determine  if  the  model  logic  is  correct  and  rite  model's  accuracy  is  maintained 
(Sargent,  1991:41).  Five  experts  agreed  to  evaluate  the  conceptual  model.  Four  of  the 
five  experts  returned  their  comment  sheets  and  were  subsequently  interviewed  (3  from 
ACC  and  1  from  ESC).  The  fifth  expert's  participation  was  constrained  by  temporary 
duty  assignments  and  leave. 

The  experts  responded  to  three  demographic  questions  designed  to  substantiate 
their  expertise.  The  experts  also  answered  two  face  validation  questions  on  the  overall 
model  and  seven  detailed  questions  to  assist  the  researchers  with  model  refinement.  The 
demographic  responses  and  results  from  the  two  face  validation  questions  are 
summarized  in  Table  4-1.  These  results,  combined  with  interview  comments  and 
answers  to  the  seven  refinement  questions,  contribute  to  our  confidence  in  the  face 
validity  of  the  conceptual  model  presented  in  this  thesis.  The  minor  disconnects  noted  in 
the  second  expert’s  response  on  model  validity  and  in  some  detailed  question  responses 
were  resolved  after  discussing  the  changes  with  the  experts  in  the  follow-up  interviews. 
The  only  modifications  a  the  overall  model  were  semantic  changes  to  more  generally 
accepted  terminology  (e.g.,  frag  breakout  instead  of  frag  generation)  and  clarification  of 
responsibilities  for  crew  scheduling.  Some  additional  data  requirements  were  added  to 
the  detailed  back-up  material.  These  changes  were  incorporated  into  the  ACMP  model 
presented  in  this  thesis.  Additionally,  the  detailed  responses  and  interview  comments 
influenced  our  conclusions  and  recommendations  discussed  in  the  final  chapter. 
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Table4-1 

Expert  Comments  On  ACMP  Model 
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Objective  #2  —  Proposed  ACMP  Effectiveness  Evaluation  Methodology 

This  thesis's  second  objective  was  to  derive  from  previous  research  a  general 
method  for  evaluating  the  process  effectiveness  of  an  ACMP  process.  As  discussed  in 
Chapter  III,  this  thesis's  work  primarily  focused  on  providing  Air  Force  operational 
commands  a  methodology  for  appraising  overall  ACMP  process  effectiveness. 
Additionally,  we  attempted  to  integrate  into  the  methodology  measures  that  differentiate 
component  contribution  to  effectiveness.  However,  this  additional  benefit  was  weighed 
against  the  additional  complexity  forced  on  the  field  evaluators.  Our  goal  was  to  identify 
methods  that  can  be  used  by  field  managers  within  the  ACMP  environment.  Our 
understanding  of  the  current  ACMP  environment  is  reflected  in  the  conceptual  model  of 
the  ACMP  process  developed  for  Objective  #1.  The  conceptual  model  may  change  and 
evolve  over  time  as  the  Air  Force  changes.  However,  since  conceptual  models  only 
provide  a  starting  point  for  an  analyst  evaluation  of  management  issues,  the  methodology 
can  be  used  with  any  ACMP  process.  Our  proposed  approach  is  shown  in  Figure  4-13. 

We  used  the  general  model  for  evaluation  discussed  by  Sprague  and  Carlson  as 
the  framework  for  deriving  our  methodology.  This  model  for  planned  experiments  is 
widely  used  across  many  fields.  It  includes  the  following  six  elements;  1)  statement  of 
objectives,  2)  choice  of  measures,  3)  choice  of  treatments  and  experimental  units.  4)  plan 
for  assigning  treatments  to  experimental  units,  5)  plan  for  selecting  the  experimental  units 
and  6)  choice  of  analysis  criteria  and  techniques  (Sprague  and  Carlson,  1982:  167). 
Therefore,  the  first  step  of  our  evaluation  approach  is  definition  of  a  management  issue. 
We  choose  to  use  the  terminology  “definition  of  a  management  issue"  instead  of 
“statement  of  objectives"  because  the  selected  terminology  may  be  more  easily 
understood  by  the  targeted  field  personnel  as  compared  to  researchers.  However,  we 
retain  the  same  intent. 
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Rgurc  4-13.  ACMP  Evaluation  Approach 


In  the  first  evaluation  methodology  step  of  defining  a  management  issue,  we 
expect  the  field  manager  to  state  as  a  hypothesis  an  ACMP  management  question  tnai  the 
organization  faces.  For  example,  a  Headquarters  Air  Combat  Command  (ACC)  manager 
might  need  to  assess  wing  comtriander  requirements  for  two  additional  computerized 
mission  planning  stations  per  wing  to  increase  the  amount  of  planning  that  can  be 
accomplished  during  combat  surges.  The  ACC  manager  seeking  to  evaluate  this  issue 
would  state  the  hypothesis  that  adding  two  additional  computerized  mission  planning 
stations  per  wing  will  increase  the  amount  of  planning  that  could  be  accomplished. 
Generally,  the  more  specific  the  hypothesis,  the  easier  the  experiment  is  to  design  and 
interpret  (Sprague  and  Carlson,  1982;  168).  Therefore,  if  evaluators  can  use  their 
knowledge  to  specify  a  required  increase,  they  should. 

The  second  element  of  evaluation  suggested  by  the  general  model  for  evaluation 
is  choice  of  measures.  The  effectiveness  of  Air  Combat  Mission  Planning  is  generally 
demonstrated  through  the  effectiveness  of  the  combat  plan’s  implementation  and  aircrew 
survival.  Therefore,  combat  effectiveness  and  survival  are  the  overall  measure  of 
planning  effectiveness.  While  there  are  several  criteria  that  contribute  to  combat 
effectiveness  and  survival  (i.e.,  weapon  system  effectiveness,  level  of  threat,  weather, 
aircrew  effectiveness,  and  plan  quality),  only  plan  quality  is  determined  by  the  ACMP 
process.  Therefore,  plan  quality  is  an  appropriate  measure  for  evaluating  ACMP 
effectiveness.  Our  ACMP  evaluation  methodology  suggests  two  vectors  for  evaluating 
plan  quality:  Behavioral  Factors  and  Mission  Factors.  These  vectors  are  derived  from 
Metersky’s  C^  Process  Evaluation  methodology  that  also  uses  two  vectors:  Generic 
Factors  and  Mission  Factors.  Metersky’s  generic  factors  are  parameters  that  are 
independent  of  the  mission  application  and  included  “receptor  and  sender  biases"  and 
“vigilance  and  fatigue."  Metersky’s  mission  vector  factors  consist  of  parameters  that  are 
unique  to  one  or  more  missions  and  included  tactics  and  available  resources  (Metersky, 
1986:887-889).  We  chose  to  call  the  first  vector  "Behavioral  Factors"  instead  of 
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“Generic  Factors’’  because  it  more  accurately  describes  the  independent  factors  within  the 
ACMP  context.  The  concept  of  separating  behavioral  and  mission  factors  has  been  used 
by  other  researchers  in  addition  to  Metersky  although  they  also  use  different  terms  to 
describe  the  vectors  (Adelman  «  u/.,  1981:61;  Adelman  and  Ulvila,  1991 :546). 

The  third  step  suggested  by  the  general  model  for  evaluation  is  choice  of 
treatments  and  experimental  units.  Within  the  context  of  ACMP  evaluation  the 
treatments  are  changes  to  the  ACMP  process  (e.g.,  addition  of  computerized  mission 
planning  stations).  Experimental  units  are  a  model  of  a  current  specific  ACMP  process 
that  serves  as  a  control  and  models  of  one  or  more  alternative  ACMP  processes  which 
receive  the  treatments  (ACMP  process  changes).  Our  ACMP  evaluation  approach 
provides  a  conceptual  model  of  the  ACMP  process  to  simplify  specific  model  generation. 
An  approach  to  specific  model  generation  is  demonstrated  in  the  third  thesis  objective. 

The  fourth  and  fifth  steps  suggested  by  the  general  model  for  evaluation,  plan  for 
assigning  the  treatments  and  sample  selection  plan,  are  simplified  in  ACMP  evaluation 
because  ail  specific  models  generated  are  involved  in  the  experiment.  This  simplification 
or  use  of  all  models  is  often  the  case  in  decision-aiding  evaluation.  In  our  example,  plan 
quality  is  compared  between  the  control  model  of  the  specific  ACMP  process  and 
treatment  models.  This  simplified  situation  is  normal  in  decision-aiding  evaluation 
(Sprague  and  Carlson,  1982:168-169). 

The  sixth  step  suggested  by  the  general  model  for  evaluation  is  choice  of  analysis 
criteria  and  techniques.  Given  that  our  methodology  evaluates  two  vectors  of  plan 
quality  (Behavioral  and  Mission),  we  considered  appropriate  measures  for  each. 

Metersky  suggests  use  of  social  sciences  techniques  such  as  questionnaires,  observations 
or  interviews  to  evaluate  the  behavioral  vector.  He  suggests  use  of  simulation,  man-in- 
the-loop  gaming,  or  operational  exercises  to  evaluate  the  mission  vector  (Metersky,  1986: 
889),  Our  objective  of  developing  an  ACMP  evaluation  approach  that  could  be  used  by 
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managers  evaluating  user  options  required  deriving  techniques  that  would  not  be  too 
expensive  or  complex.  We  considered  the  mission  vector  first. 

While  there  may  be  situations  when  man-in-the-ioop  gaming  or  operational 
exercises  could  be  used  to  compare  ACMP  processes,  for  the  most  part  these  would  be 
too  expensive  and  complex  to  set-up.  This  conclu.sion  is  supported  by  discussions  with 
target  evaluation  approach  users  at  HQ  ACC.  As  shown  in  the  ACMP  Evaluation 
Approach  (Figure  4-13),  simulation  experiments  would  be  set  up  to  compare  selected 
factors  that  contribute  to  plan  quality.  Factors  are  selected  based  on  their  relevance  to  the 
management  issue.  While  most  of  the  selected  factors  would  be  mission  factors  that  are 
easily  quantified  (e.g.,  study  time,  preparation  time,  and  number  of  plans),  for  some 
management  questions  it  might  be  appropriate  to  operationalize  behavioral  factors.  For 
example,  while  the  effects  of  fatigue  could  not  be  directly  simulated  and  measured,  the 
user  could  “operationalize"  evaluation  of  fatigue  by  assuming  factors  for  lengthening 
planning  times  due  to  hypothesized  fatigue  situations.  The  strength  of  inferences  drawn 
from  the  “operationalized"  factors  depends  on  how  strongly  and  directly  these  factors  can 
be  related  to  operational  measures.  The  result  from  comparing  selected  factors,  or  testing 
a  hypothesis  about  differences  between  specific  models,  is  a  proposed  management 
solution.  However,  before  the  evaluation  is  complete,  the  behavioral  vector  must  be 
considered. 

As  discussed  in  the  literature  review,  the  most  appropriate  approach  for 
synthesizing  quantitative  (Mission  Vector)  and  qualitative  factors  (Behavioral  Vector)  is 
use  of  an  appropriate  systemic  approach  for  weighting  facts  and  values  (e.g.,  Delphi  or 
Multiattribute  Utility  Theory).  Developing  these  weights  would  be  complicated  by  our 
lack  of  understanding  of  how  the  mission  and  behavioral  vectors  interact  in  the  ACMP 
environment.  The  situation  is  further  complicated  by  our  inability  to  infer  or  assume 
general  relationships.  Our  literature  review  revealed  multiple  decision-making  models  in 
the  two  broad  and  fundamentally  differing  categories  of  normative  and  descriptive 


4-27 


models.  Further,  as  Wohl  found,  “we  are  barely  scratching  the  surface  of  understanding 
the  process  of  tactical  decision-making”  (Wohl,  1981:31).  Within  the  context  of  ACMP 
evaluation,  we  found  it  unrealistic  to  assume  the  user  would  be  able  to  develop 
statistically  relevant  weights  that  would  relate  the  significance  of  behavioral  factors 
(previously  presented  in  Table  2-1)  to  mission  factors  (planning  time,  study  time,  etc.). 
The  lack  of  a  foundation  of  basic  research  into  these  relationships  severely  limits  the 
ability  of  the  user  to  develop  these  weights  through  either  querying  experts  or  conducting 
experiments.  Because  these  limitations  preclude  use  of  the  preferred  systemic 
methodology,  alternatives  must  be  considered.  If  a  specific  behavioral  factor  is 
considered  significant  to  the  management  issue,  we  recommend  the  user  attempt  to 
operationalize  the  factor  into  the  simulation  as  previously  discussed.  If  behavioral  factors 
are  not  a  direct  issue,  we  recommend  the  users  consider  these  factors  during  the 
management  evaluation  of  the  proposed  solution.  While  we  found  no  bases  for  formally 
evaluating  the  behavioral  factors,  we  believe  they  are  increasingly  important  and  must  not 
be  overlooked.  We  recommend  improving  this  situation  in  Chapter  V. 

Objective  #3  -  Demonstrate  Application  of  ACMP  Evaluation  Method 

Our  final  research  objective  was  to  demonstrate  the  application  of  our  conceptual 
If*  !«1  and  our  ACMP  evaluation  methodology  on  a  specific  management  issue.  The 
purpose  of  the  demonstration  was  to  test  the  evaluation  methodology's  reasonableness 
and  to  illustrate  its  use  to  the  reader.  To  perform  this  demonstration,  we  started  with 
three  constructs:  1)  a  management  issue  (represented  by  our  problem  scenario  discussed 
in  Chapter  III),  2)  our  conceptual  model  of  the  ACMP  process,  and  3)  our  proposed 
evaluation  methodology  discussed  under  the  results  of  research  objective  M2  and  depicted 
in  Figure  4-13. 

Following  the  steps  of  our  evaluation  methodology,  we  selected  our  mission 
factors  of  plan  quality  and  developed  a  specific  model  of  the  “current”  ACMP  process. 
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This  specific  model  was  developed  from  the  problem  scenario  and  programmed  in  SLAM 
II  using  the  conceptual  ACMP  model  as  a  starting  point.  Alternatives  to  the  “current” 
process  were  also  modeled  in  SLAM  II  for  comparison  against  the  baseline  model  of  the 
“current”  system.  Each  alternative  model  was  simulated  and  data  on  the  performance 
measures  were  collected  for  analysis  and  comparison.  After  generation  of  the  data, 
graphical  analysis  methods  were  used  to  compare  the  model  alternatives  and  identify 
candidate  answers  to  the  management  issue.  Finally,  the  effects  of  behavioral  factors  not 
included  or  operationalized  in  the  simulation  models  were  considered  and  an  answer  or 
solution  to  the  problem  presented.  The  following  sections  provide  more  detail  on  the 
results  of  this  demonstration. 

Evaluation  Step  #1  -  Define  the  Management  Issue.  The  first  step  of  our 
evaluation  approach  was  to  define  clearly  the  issue  or  problem  the  manager  was  trying  to 
solve.  HQ  ACC  wanted  to  determine  how  large  an  increase  in  the  time  required  for  flight 
planning  could  be  before  plan  quality  began  to  suffer.  In  this  demonstration  the  issue 
was  easy  to  identify  because  it  was  provided  as  part  of  the  problem  scenario.  In  practice, 
trying  to  identify  the  real  problem  or  question  can  be  difficult  and  requires  a  working 
knowledge  of  the  process  or  system  under  study.  The  conceptual  model  established  as 
part  of  this  thesis  can  be  of  some  help  to  those  trying  to  understand  the  process. 

Our  methodology  proposes  that  the  management  issue  be  stated  in  the  form  of  a 
hypothesis,  or  set  of  hypotheses  that  can  be  tested  using  specific  models.  Stating 
hypotheses  helps  narrow  the  issue  for  analysis  and  decision.  For  our  demonstration,  we 
stated  the  following  set  of  hypotheses: 

Hypothesis  A.  Adding  5  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 

Hypothesis  B.  Adding  10  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 

Hypothesis  C.  Adding  15  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 
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Hypothesis  D.  Adding  20  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 

Hypothesis  E.  Adding  25  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 

Hypothesis  F.  Adding  30  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 

Hypothesis  G.  Adding  35  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  a^ect  the  quality  of  our  mission  plans. 

Hypothesis  H.  Adding  40  minutes  to  the  average  time  needed  to  prepare  flight  plans  on 
mission  planning  stations  does  not  adversely  affect  the  quality  of  our  mission  plans. 

The  question  “how  much  time  can  be  added  to  a  flight's  planning  time  before  plan 
quality  begins  to  suffer?”  can  be  ansv.'ered  by  determining  the  point  at  which  we  begin 
rejecting  the  hypotheses.  For  instance,  if  we  fail  to  reject  hypotheses  A.  B,  C,  and  D,  but 
reject  E,  F,  G,  and  H,  it  suggests  that  at  least  20  minutes  can  be  added  to  the  current 
planning  time  without  adversely  affecting  the  quality  of  our  mission  plans. 

Evaluation  Step  #2  --  Choice  of  Measures.  The  second  step  in  our 
methodology  was  to  determine  a  measure  or  set  of  measures  that  could  be  used  to 
compare  the  models  and  test  the  hypotheses.  As  suggested  from  our  research,  a  basic 
measure  of  the  mission  planning  process  effectiveness  is  plan  quality.  This  measure 
presents  a  problem  because  plan  quality  nnot  be  directly  measured.  Our  research 
indicated,  and  therefor^,  our  evaluation  approach  suggests,  that  plan  quality  can  be 
assessed  by  attempting  to  measure  separately  its  mission  vector  ai.d  its  behavioral  vector. 

The  problem  scenario  specified  several  mission  vector  factors.  These  were 
measures  that  the  decision-makers  thought  were  important  in  determining  the  quality  of 
the  mission  plans.  The  first  measure  was  the  number  of  incomplete  flight  plans  generated 
due  to  insufficient  planning  time  (uumber  of  incomplete  plans).  We  generally  assume 
that  plan  quality  suffers  ii'the  planners  do  not  have  sufficient  time  to  complete  all  of  their 
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planning  tasks.  The  number  of  incomplete  plans  that  occur  over  some  period  of  time 
indicates  how  well  the  mission  planning  process  is  functioning. 

The  other  measures  of  plan  quality  are  time  measurements.  They  are  closely 
related  to  each  other  and  involve  measuring  the  actual  time  spent  flight  planning  at  the 
planning  stations  (flight  planning  time),  the  amount  of  time  available  from  completion  of 
flight  planning  until  step  time  (slack  time),  and  the  amount  of  planning  time  the  planners 
did  not  receive  because  they  tan  out  of  time  (lost  planning  time).  These  measures  are 
illustrated  in  Figure  4-14.  The  amount  of  actual  flight  planning  time  and  lost  planning 
time  help  indicate  whether  planning  time  is  adequate,  and  the  severity  of  a  problem  if  it 
exists.  For  this  demonstration,  we  assumed  that  plan  quality  and  ntission  effectiveness 
are  enhanced  if  the  crews  have  time  to  study  the  plan  before  execution.  The  time  crews 
have  for  studying  is  represented  by  slack  time  values. 

Evaluation  Step  #3  —  Building  the  Model  and  Selecting  Treatment 
Levels.  Our  choice  of  treatments  was  determined  by  the  hypothestis  we  defined  in  the 
first  step  of  our  demonstration.  The  approach  involved  the  construction  of  nine  simulation 
models  (a  baseline  model  representing  the  “current”  process  and  one  for  each  hypothesis) 
where  the  average  flight  planning  time  used  in  each  model  reflected  the  amount  of  time  to 
be  tested  by  the  hypothesis.  Common  random  numbers  were  used  between  the  models. 
The  only  differenr ;  between  models  was  the  flight  planning  time  distribution  used  for 
assigning  planning  times  to  each  flight.  For  instance,  the  baseline  model  applied  a 
random  uniform  distribution  with  flight  planning  times  between  10  to  20  minutes.  Model 
A  (corresponding  to  hypothesis  A)  applied  a  random  uniform  distribution  with  flight 
planning  times  between  15  to  25  minutes  to  represent  an  addition  of  5  minutes  to  the 
average  flight  planning  time. 

The  models  developed  for  this  demonstration  were  combined  network  and 
discrete  event  models  programmed  in  SLAM  II  and  FORTRAN.  The  mission  planning 
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Available  Time  Ground  Operations 
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Figure  4- 14.  niDstratkin  of  Performance  M( 


process  models  were  built  to  represent  the  mission  planning  process  activities,  times,  and 
priorities  as  stated  in  the  problem  scenario.  A  detailed  description  of  those  models  is  not 
included  as  part  of  this  thesis.  However,  a  commented  FORTRAN  source  code  listing  for 
the  discrete  events,  a  graphical  depiction  of  the  SLAM  II  network,  and  the  SLAM  II 
program  code  for  each  model  version  is  provided  in  Appendix  D. 

Evaluation  Step  #4  -  Simulation  and  Analysis  Techniques.  Once  the 
specific  models  were  built  and  verified,  pilot-runs  and  then  simulation  experiments  were 
conducted.  Each  run  or  replication  of  the  model  simulated  one  crew  day  of  planning. 
During  each  run  of  a  model,  six  flights  attempted  to  plan  and  fly  two  missions  during  the 
crew  day.  Each  run  produced  data  on  a  total  of  twelve  flight  plans  (six  during  planning 
session  one,  and  six  during  planning  session  two).  We  determined  through  pilot-runs  of 
the  baseline  model  that  the  run-time  of  the  model  was  short.  Because  long  run-times 
were  not  a  major  factor  in  conducting  the  simulation,  each  model  was  replicated  1(X) 
times.  This  number  of  replications  provided  a  simulation  of  1(X)  crew  days  and  data  on 
over  1200  individual  flight  planning  attempts.  We  considered  this  data  more  than 
sufficient  for  this  demonstration. 

The  number  of  incomplete  plans,  average  flight  planning  time,  average  slack  time, 
and  average  lost  time  was  collected  by  planning  session  for  each  replication  of  each 
model.  After  collecting  the  data,  the  results  were  compiled  and  prepared  for  graphical 
analysis.  Graphical  analysis  was  chosen  because  preliminary  analysis  of  the  data 
indicated  that  for  this  specific  scenario,  model  differences  were  easily  detected  and 
illustrated  using  this  approach.  Second,  the  use  of  graphical  techniques  could  be  more 
easily  briefed  to,  and  understood  by,  our  research  sponsors  (the  decision-makers).  Thus, 
a  graphical  approach  provided  an  adequate,  yet  less  complex  and  more  presentable 
approach  than  a  more  fonnal  statistici.l  approach  to  data  analysis.  We  do  not  suggest  that 
the  graphical  approach  will  always  be  ad'*quatc.  Depending  upon  the  issue,  performance 
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measures,  model  detail,  and  complexity  of  the  experimental  design,  statistical  techniques 
may  be  required  and  preferred. 

Simulation  Results.  Results  from  the  simulation  experiments  are 
summarized  in  Tables  4-2  through  4-5,  and  Figures  4-15  through  4-17  When  we  looked 
at  the  number  of  incomplete  plans  (Table  4-2  and  Figure  4-15)  recorded  for  each  model, 
we  saw  that  the  baseline  model  could  support  the  planning  load  without  requiring  any  of 
the  planners  to  halt  planning  to  meet  their  take-off  time.  Alter  analyzing  the  results  from 
each  successive  model,  we  noted  that  the  data  indicates  that  a  change  begins  to  take  place 
in  the  system  between  the  runs  associated  with  model  B  and  model  C.  Model  B,  which 
added  10  minutes  to  the  average  planning  time  of  the  current  system,  could  support 
planning  without  generating  incomplete  plans.  However,  the  data  from  model  C,  which 
added  15  minutes  to  the  average  planning  time  of  the  current  system,  indicates  the 
occurrence  of  incomplete  plans.  Data  from  models  D,  E,  F,  G,  and  H  show  that  as  more 
time  is  added  to  the  planning  duration,  the  number  of  incomplete  plans  continues  to 
increase. 

When  reviewing  the  data  on  actual  flight  planning  times  (Table  4-3  and  Figure  4- 
16),  we  saw  a  noticeable  change  taking  place  between  models  C  and  D.  As  planning  time 
increases  linearly  (-t-5  minutes)  from  model  to  model,  the  actual  flight  planning  times  also 
appear  to  increase  in  a  linear  fashion  from  the  baseline  model  through  mode!  C. 

However,  a  distinct  change  occurs  in  the  system  as  we  move  between  models  C  and  D. 
This  change  is  depicted  by  the  change  in  slope  of  the  line  connecting  the  data  points. 
Beginning  with  model  D,  the  average  flight  planning  time  appears  to  level  out,  although 
the  input  planning  time.i  are  increased  with  each  successive  model.  As  we  determined  b  v 
looking  at  slack  time  and  lost  planning  time,  this  change  in  average  flight  planning  times 
is  caused  by  the  occurrence  of  incomplete  plans  that  offset  the  increase  in  planning  time 
we  introduce  with  each  new  model. 


4-34 


Resales  frem  Stm 


4-35 


Each  replication  attempted  six  fli^t  plans  per  session. 

Results  are  based  on  600  plans  per  session  or  1200  total  plans. 


INCOMPLETE  PLANS 

(By  Model  and  Session) 
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flfare  4-15.  Number  of  Incomfrfete  i^ans  Recorded  by  Model  and  Session 
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AVERAGE  ACTUAL  PLANNING 

(By  Model  and  Session) 
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Resahs  frsai  SuialafiMi  Expcr»cafs 
(Slack  Tine) 
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Table  4*5 


SLACK/LOST  PLANNING  TIME 

(By  Model  and  Session) 
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figure  4-17.  Average  Slack  Time  and  Average  Lest  Time  Results  Modd  and  Session 


When  we  reviewed  the  slack  time  (Table  4-4)  and  lost  planning  time  data  (Table 
4'5),  we  saw  that  system  changes  appear  to  occur  either  between  models  C  and  D  (using 
slack  time  as  the  measure)  or  models  B  and  C  (using  lost  planning  time  as  the  measure). 
Slack  time  appears  to  decrease  linearly  until  model  C  and  then  experiences  a  change  in 
slope  as  we  track  the  line  beyond  model  D  (Figure  4-17).  With  lost  planning  time  as  the 
measure,  the  change  occurs  between  models  B  and  C  as  incomplete  flight  plans  develop 
and  lost  planning  times  are  collected  (Figure  4-17).  Thus,  our  four  performance  measures 
appear  to  give  conflicting  results.  Closer  examination  reveals  however,  that  this 
discrepancy  is  a  function  only  of  the  accuracy  of  our  graphical  technique. 

While  not  noticeable  on  the  graph,  a  change  in  the  rate  of  change  of  average 
actual  flight  planning  time  can  be  detected  between  model  B  and  C.  We  found  this  slight 
change  while  trying  to  resolve  the  apparent  difference  given  by  the  measure  of  average 
flight  planning  time  (which  first  graphically  indicated  a  change  between  models  C  and  D) 
and  the  other  performance  measures  (which  graphically  indicated  a  change  between 
models  B  and  C).  This  example  shows  that  it  is  sometimes  helpful  to  use  more  than  one 
related  performance  measure  so  cross  checks  can  be  made  of  the  results.  This 
discrepancy  also  highlights  a  known  weakness  with  the  graphical  approach.  The 
graphical  approach  is  sometimes  not  sensitive  enough  to  detect  small,  but  statistically 
significant,  differences. 

As  a  result  of  our  simulation  experiments,  we  concluded  that  with  planning  times 
between  20-30  minutes  (a  10  minute  increase  over  the  current  planning  time),  planners 
are  able  to  complete  planning  without  finishing  prematurely.  On  the  other  hand,  when 
planning  times  are  between  25  ••  35  minutes  (a  15  minute  increase  over  the  current 
average  planning  time),  plans  are  occasionally  incomplete. 

Before  we  could  state  these  simulation  results  as  our  final  solution  to  the  problem, 
our  methodology  required  us  to  conduct  a  management  evaluation  of  our  preliminary 
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results  considering  the  behavioral  factors  of  plan  quality  to  determine  if  they  might 
change  our  findings.  We  assumed,  based  on  our  research  findings,  that  management 
lacked  specific  data  on  the  relative  importance  of  behavioral  factors.  Lacking  any 
specific  basis  for  selecting  behavioral  factors,  we  subjectively  chose  the  following 
behavioral  factors  for  consideration:  receptor  and  sender  biases;  vigilance;  fatigue;  and 
content,  clarity,  and  appropriate  use  of  information  input.  Unfortunately,  as  indicated  in 
our  discussion  of  the  methodology,  we  could  not  derive  an  objective  approach  to 
assessing  the  effects  of  these  behavioral  factors  that  would  be  practical  for  our  target 
model  users.  In  place  of  an  objective  approach,  we  instead  tried  to  determine  subjectively 
whether  each  of  the  factors  would  tend  to  increase  or  decrease  plan  quality,  and  by  how 
much  they  would  change  the  quality  of  the  plan.  As  a  result  of  that  determination,  our 
management  evaluation  concluded  that  vigilance  could  increase  the  quality  of  plans 
produced  during  a  mission  planning  session.  However,  the  remaining  behavioral  factors 
all  tended  to  detract  from  the  plan  quality.  Given  that  our  methodology  does  not  provide 
a  means  for  establishing  relationships  between  behavioral  factors,  we  were  unable  to 
determine  how  much  each  of  the  factors  might  contribute  to  a  change  in  plan  quality,  and 
under  what  circumstances  those  changes  might  take  place. 

Our  assessment  of  how  behavioral  factors  alfect  the  mission  planning  process  did 
not  provide  any  quantitative  results  that  might  be  applied  to  the  simulation  results.  At 
best,  we  might  assume  that  most  behavioral  factors  would  detract  from  each  of  the 
models  equally,  and  therefore  not  affect  the  relative  relation  between  the  models.  Our 
lack  of  an  objective  method  for  assessing  the  effects  of  the  behavioral  factors  prevented 
us  from  quantifying,  and  thus  integrating  the  mission  and  behavioral  aspects  of  plan 
quality.  Because  of  that  problem,  the  results  of  our  evaluation  are  based  upon  our 
simulation  experiments. 
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In  conclusion,  because  we  assumed  earlier  that  the  occuirence  of  incomplete  plans 
constitutes  a  decrease  in  the  qualit^i  of  those  plans,  we  reached  the  following  conclusions 
with  respect  to  our  hypotheses: 


Hypothesis  A  (+5  minutes)  -  Fail  to  reject 

Hypothesis  B  (•«•  10  minutes)  -  Fail  to  reject 

Hypothesis  C  (+ 1 5  minutes)  --  Reject 
Hypothesis  D  (+20  minutes)  -  Reject 
Hypothesis  E  (+25  minutes)  —  Reject 
Hypothesis  F  (+30  minutes)  —  Reject 
Hypothesis  G  (+35  minutes)  -  Reject 
Hypothesis  H  (+40  minutes)  -  Reject 


We  conclude  from  cur  evaluation  that  the  largest  amount  of  time  that  can  be 
added  to  the  current  planning  process  without  affecting  the  quality  of  the  resulting  plans 
is  ten  minutes. 

Summary  —  Objective  ^3.  The  demonstration  produced  several  results.  First, 
we  were  able  to  successfully  build  specific  models  of  a  mission  planning  process  using 
our  conceptual  model  as  a  starting  point.  We  believe  this  helps  substantiate  the  validity 
claim  about  the  usefulness  of  our  conceptual  model  as  a  starting  point  for  construction  of 
the  ACMP  process.  Second,  it  showed  that  at  least  in  this  case,  the  evaluation 
methodology  is  capable  of  helping  a  decision  maker  reach  a  proposed  solution.  While 
further  research  and  demonstrations  of  the  methodology's  use  are  needed  before  any 
absolute  claims  of  validity  can  be  made  about  our  approach,  we  were  nonetheless  pleased 
to  be  able  to  reach  a  proposed  solution.  Finally,  the  demonstration  highlighted  a 
weakness  in  our  methodology.  While  the  use  of  simulation  and  models  appears  to  allow 
for  the  measurement  of  many  mission  or  performance  aspects  of  plan  quality,  the 
methodology  fails  to  provide  a  nonsubjective  approach  for  evaluating  and  integrating  the 
behavioral  aspects  of  plan  quality. 
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V.  Conclusions  and  Recommendations 


Overview 

This  chapter  summarizes  conclusions  and  recommendations  from  our  research.  It 
begins  with  a  sununary  of  research  that  reviews  our  objectives,  literature  review,  and 
findings  from  our  research.  The  chapter  continues  with  a  discussion  of  major  conclusions 
from  this  research  and  concludes  with  recommended  follow-on  research. 

Summary  of  Research 

The  goal  of  our  research  was  to  identify  or  develop  an  appropriate  method  for 
evaluating  the  effectiveness  of  unit-level  Air  Combat  Mission  Planning  (ACMP)  process. 
We  chose  this  goal  because  published  research,  reports,  and  our  experience  led  us  to 
believe  that  developing  the  best  possible  ACMP  process  necessitates  rigorous  evaluation 
of  alternatives  to  ensure  effectiveness  is  either  maintained  or  improved.  We  suggest  that 
as  the  Air  Force  orf^anizationally  restructures  the  unit  level,  automates  the  ACMP 
process,  and  add*:  high-technology  weapons,  the  need  for  a  structured  evaluation 
methodology  to  support  decision  makers  will  increase.  We  approached  our  research  goal 
by  identifying  three  related  research  objectives. 

Research  Objective  #1.  Define  a  conceptual  or  “generic”  Air  Combat  Mission 
Planning  (ACMP)  process  and  select,  or  adapt,  an  existing  C3  or  combat  planning  model 
that  accurately  reflects  the  defined  ACMP  process. 

Research  Objective  #2.  Derive  from  previous  research  a  general  method  for 
evaluating  the  process  effectiveness  of  an  ACMP  process. 

Research  Objective  Demonstmte  application  of  the  conceptual  ACMP  model 
resulting  from  Research  Objective  #1,  end  the  general  ACMP  process  evaluation  method 
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developed  in  Research  Objective  #2,  to  the  investigation  of  a  specific  ACMP 
management  issue. 

In  Chapter  II’s  literature  review,  we  introduced  our  operational  topics--a 
walkthrough  of  the  ACMP  process  and  the  results  of  our  research  on  mission  planning 
lessons  learned  from  Desert  Storm.  Chapter  II  also  reviewed  a  wide  range  of  academic 
literature  that  provides  a  foundation  for  ACMP  evaluation  despite  the  absence  of  specific 
research  on  our  topic.  Figure  2-2  showed  the  parallels  between  the  trends  identified  in 
our  operational  review  and  findings  from  academic  review.  Many  of  the  parallel  trends 
and  findings  reflect  the  need  to  improve  the  ACMP  process.  Other  trends  and  findings 
reflect  the  increasing  complexity  of  the  ACMP  process  and  the  complexity  of  C^ 
evaluation.  These  results  support  the  need  for  an  ACMP  evaluation  methodology. 

Additionally,  the  academic  findings  support  evaluating  an  ACMP  process  by 
considering  plan  quality.  We  found  research  indicating  that  plan  quality  should  be 
measured  on  two  vectors-behaviural  factors  and  mission  factors.  Mission  factors  (e.g., 
availability  of  data,  availability  of  resources,  and  performance)  can  be  measured  with 
simulations  or  exercises.  Academic  researchers  suggest  behavioral  factors  (previously 
presented  in  Table  2-1)  are  best  measured  individually  with  social  science  tools  .  The 
individual  measures  should  be  integrated  by  summing  weighted  results.  However,  there 
should  be  some  experimental  or  expert  basis  for  the  weights.  Unfortunately,  we  found  no 
basis  for  accurately  weighting  the  relative  importance  of  behavioral  factors  involved  in 
the  ACMP  process. 

Chapter  III  discusses  our  methodologies  for  researching  each  of  our  three 
objectives.  The  methodologies,  while  distinct  for  each  objective,  converge  on  model - 
based  approaches  to  problem  solving.  The  research  foundation  for  our  proposed  model- 
based  research  approach  is  presented  in  Chapter  III.  Pritsker’s  model-based  problem 
solving  apprt.  ich  di.scusscd  in  Chapter  III,  and  Sprague  and  Carlson's  general  model  for 
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evaluation  discussed  in  Chapter  IV,  both  derive  their  fundamental  strength  from  universal 
scientific  methods  of  research. 

We  used  both  Pritsker’s  approach  and  Sprague  and  Carlson’s  general  model  to 
form  our  proposed  ACMP  Effectiveness  Evaluation  Methodology  (previously  shown  in 
Figure  4-13).  Our  methodology  is  a  tool  that  can  be  used  by  management  to  evaluate  the 
effectiveness  of  proposed  alternatives  to  the  ACMP  process.  The  general  model  for 
evaluation  provided  an  accepted  framework  for  conducting  evaluation.  The  model-based 
problem  solving  approach  provided  a  means  of  gaining  increased  understanding  of  the 
ACMP  process  and  an  approach  to  using  models  for  addressing  management  problems  in 
systems  where  comparison  of  actual  processes  is  not  practical.  Because  it  is  difficult  to 
replicate  the  ACMP  process  during  peacetime,  ACMP  process  experimentation  is  not 
practical  except  on  models.  For  that  reason,  we  considered  our  methodology  based  on 
the  general  model  for  evaluation  and  the  model-based  problem  solving  approach  to  be 
appropriate. 

Expert  and  sponsor  acceptance  of  the  conceptual  model  and  of  the  evaluation 
methodology  provided  strong  evidence  of  the  validity  of  our  research.  We  further 
substantiated  the  validity  of  our  research  by  applying  our  proposed  ACMP  Effectiveness 
Evaluation  Methodology  to  a  real-world  management  issue  suggested  by  our  sponsors. 
While  Objective  #3’s  demonstration  was  limited  by  available  resources  and  time,  we 
believe  operational  command  evaluation  is  often  similarly  limited.  Our  conclusions  from 
this  demonstration  and  the  overall  research  are  summarized  in  the  next  section. 

Conclusions 

Despite  the  limitations  of  the  proposed  ACMP  EiTectiveness  Evaluation 
Methodology  that  we  will  emphasize  in  this  section,  we  are  convinced  that  our  research 
objectives  and  goals  have  been  satisfied.  We  developed  a  conceptual  or  ‘generic”  model 
of  the  ACMP  process  using  generally  accepted  modeling  concepts  previously  cited.  The 
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model  was  based  on  a  solid  foundatic  :i  of  published  material  reviewed  in  the  operational 
section  of  Chapter  II,  interviews  with  Air  Force  personnel,  and  our  prior  experiences  with 
mission  planning  activities.  Our  ACC  and  ESC  sponsors  have  both  indicated  that  the 
conceptual  model  we  developed  represents  a  necessary  and  useful  starting  point  for 
achieving  greater  Air  Force  understanding  of  the  ACMP  process  and  the  steps  necessary 
for  ACMP  process  improvement. 

Through  our  review  of  academic  literature,  we  sought  to  identify  a  methodology 
for  evaluating  the  effectiveness  of  an  ACMP  process.  Unfortunately,  we  discovered  no 
single  method  that  satisfactorily  simptified  the  complexities  of  planning,  decision¬ 
making,  and  decision-aiding  so  that  they  could  reasonably  be  evaluated  by  field 
personnel.  Therefore,  we  derived  an  ACMP  Evaluation  Methodology  (Figure  4-13)  from 
the  work  of  many  earlier  researchers.  However,  despite  the  use  of  supporting  research 
from  many  authors,  the  ACMP  Evaluation  Methodology  principally  relies  on  concepts 
derived  from  general  scientific  methods  of  experimentation  and  the  principle,  most 
clearly  stated  by  Metersky,  that  behavioral  and  mission  factors  in  decision  making 
can  be  evaluated  separately. 

Our  separation  of  beh/*.vioral  and  mission  factors  strengthens  the  methodology 
because  mission  factor  are  accurately  measured  while  behavioral  factors  are  subjectively 
evaluated  separately.  Unfortunately,  a  weakness  of  our  approach  is  that  it  does  not 
include  a  formal  methodology  for  the  measurement  of  behavioral  factors.  Academic 
research  suggests  two  possible  methods  of  behavioral  factor  measurement;  experiments 
and  expert  weighting  of  separately  measured  behavioral  factors.  We  believe  neither 
approach  is  currently  practical  in  the  ACMP  environment.  Conflicting  academic  theory, 
conflicting  research  results,  and  inadequate  experimental  research  on  behavioral  factors 
contribute  to  our  lack  of  confidence  in  the  ability  of  experts  to  accurately  weight 
behavioral  factors  in  an  ACMP  environment.  We  arc  even  less  confident  that  the  targeted 
users  of  our  methodology,  operational  Air  Force  personnel,  can  accurately  weight 
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behavioral  factors.  Nonetheless,  our  current  inability  to  define  an  adequate  procedural 
basis  for  evaluating  behavioral  factors  does  not  diminish  their  importance.  In  fact,  both 
our  operational  trends  and  academic  findings  CFigure  2-2)  suggest  behavioral  factors  are 
becoming  more  important. 

We  have  conceptualized  the  increasing  importance  of  behavioral  factors  as  shown 
in  Figure  5- 1.  We  believe  the  Air  Force  has  proceeded  on  the  time  continuum  along  the 
right  side  of  the  “quality  water  glass”  with  various  efforts  to  improve  ACMP  quality.  At 
first,  these  changes  were  simple  and  involved  barely  more  than  adding  automation  to 

Notional  Quality/Behavioral  Factors  Relationship 

More  Important  More  Quality 

Decentralization  of  Force  Coord 
Increased  Tempo 
Future  Smart  Munition  Pianning 
LO  Management 
Planning  Cells/Groups 
Decision  Aids  (e.g.,  Penaids) 
Increase  Data  Availability 
Digitized  Maps 
Generate  Materials 
Automate  Calculations 

Bottom  Line:  For  Maximum  Quality,  USAF  Must  Address  Behavioral  Factors 

Figure  5-1.  Notional  Qnallty/Behavioral  Factors  Water  Glass 
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minor  tasks  within  mission  planning.  However  as  we  took  steps  to  assist  planners  and 
decision-makers  with  the  increased  challenges  in  planning,  these  more  complicated  steps 
toward  increased  quality  (e.g.,  use  of  planning  cells,  decentralization)  increased  the 
importance  of  considering  behavioral  factors.  If  we  are  to  fill  the  “quality  water  glass,“ 
we  must  understand  how  our  changes  to  the  process  affect,  and  are  affected  by  behavioral 
factors.  For  that  reason,  we  recommend  additional  research  into  ACMP  behavioral 
factors  in  the  next  section. 

Another  difficulty  with  the  ACMP  Evaluation  Methodology  is  the  requirement  for 
operational  evaluators  to  balance  desires  for  model  simplicity  with  the  need  for  adequate 
representation  of  management  issue  factors.  Time  and  resource  constraints  suggest  that 
model-based  experiments  be  kept  as  simple  as  possible.  However,  the  inherent 
complexity  of  the  ACMP  process  limits  the  range  of  management  issues  that  can  be 
accurately  addressed  with  relatively  simple  models.  Our  experience  witli  this  research 
suggests  it  is  more  likely  that  ACC  will  require  increasingly  complex  analysis  to  better 
define  its  requirements.  Our  experience  while  working  with  ACC  to  develop  the 
relatively  simple  scenario  used  in  our  demonstration  illustrated  to  us  the  many  complex 
sources  of  process  variation  that  exist  in  the  ACMP  process.  These  sources  of  process 
variation  involve  differing  mission  and  aircraft  types,  the  number  and  type  of  available 
planning  resources,  differing  squadron  procedures,  a  multitude  of  behavioral  factors  and 
even  policy  decisions  that  all  increase  the  complexity  of  analysis.  The  inherent 
complexity  of  this  analysis  suggests  that  HQ  ACC  consider  obtaining  outside  assistance. 

This  level  of  relative  complexity  should  not,  however,  inhibit  use  of  the 
evaluation  methodology.  Our  greater  understanding  of  the  growing  complexity  and  pace 
of  change  in  the  mission  planning  process  combined  with  our  demonstration  results 
strongly  support  the  need  for  application  of  the  methodology.  If  unit-level  mission 
planning  requirements  are  allowed  to  continue  growing  unconstrained  by  the  Air  Force's 
ability  to  change  and  adapt  the  planning  process,  then  unit-level  mission  planning  quality 
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will  diminish  and  operational  capability  will  be  impacted.  While  only  the  F-1 17’s  Desert 
Storm  mission  planning  difficulties  were  highlighted  to  Congress,  many  of  the 
technologies  w'hich  complicate  F-1 17  planning  are  being  added  to  other  systems  (e.g., 
low  observables,  smart-munitions,  and  on-board  moving  map  displays).  Our 
demonstration  in  this  thesis  of  the  ACMP  Evaluation  Methodology  showed  the  effects  of 
adding  mission  planning  tasks  can  be  assessed. 

Despite  the  relative  complexity  of  the  ACMP  process  as  it  is  presented  in  the 
conceptual  model  and  the  completeness  of  the  model  within  the  scope  agreed  to  by  our 
sponsors,  the  utility  of  the  conceptual  model  is  limited  by  its  scope.  Several  of  our  expert 
reviewers  suggested  it  would  be  useful  to  develop  various  extensions  to  the  model. 
Although  we  believe,  and  they  agree,  that  the  conceptual  model  is  already  useful,  we  all 
recognize  that  it  is  only  the  first  step.  Some  suggested  extensions  would  add  breadth  to 
the  conceptual  model  by  adding  force-level  planning,  and  planning-related  interactions  at 
the  unit  level  after  execution  (e.g.,  debrief  and  battle  damage  assessment).  Other 
suggested  extensions  would  add  depth  to  the  model  by  exhaustively  defining  data 
requirements  and  discussing  more  variations  to  the  planning  process  (e.g.,  mass 
intelligence  briefs  are  sometimes  replaced  by  crew  briefs  to  individual  flights).  While  the 
conceptual  model  we  developed  is  not  exhaustive,  it  does  satisfy  the  requirement  of 
providing  a  substantial  foundation  for  development  of  specific  models  for  evaluation  of 
specific  management  Issues.  Nevertheless,  it  does  not  relieve  the  modeler  from  the  need 
to  validate  the  accuracy  of  the  specific  model  relative  to  the  management  issue  evaluated. 
Adding  breadth  and  depth  to  the  conceptual  model  will  further  simplify  this  task,  but  can 
never  eliminate  the  need  for  modeler  to  identify  relevant  factors.  The  ACMP 
environment  is  too  diverse  and  changing. 
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Recommendations  for  Future  Research 


Even  though  our  research  goals  and  objectives  were  satisfied,  we  have  already 
alluded  to  other  research  that  would  assist  the  Air  Force  in  improving  ACMP  processes. 
The  most  critical  of  these  is  the  need  for  basic  research  into  the  behavioral  factors  that 
influence  ACMP.  We  recommend  that  ACC  and  ESC  jointly  seek  basic  research  that 
will  identify  the  relative  importance  of  behavioral  factors  in  air  combat  mission  planning 
and  appropriate  methods  for  evaluating  impacts  of  changes  on  these  factors.  While 
behavioral  research  may  not  be  within  the  scope  of  the  current  Air  Force  Mission 
Planning  Program,  ACC  and  ESC  could  seek  additional  research  from  an  Air  Force 
laboratory,  government  research  center,  or  an  outside  contractor.  In  any  case,  the 
growing  importance  of  behavioral  factors  in  mission  planning  suggests  that  future 
improvements  in  unit-level  mission  planning  quality  may  depend  on  specific 
consideration  of  behavioral  factors. 

Other  follow-on  studies  suggested  by  our  conclusions  might  reasonably  be 
conducted  by  other  graduate  students.  Either  extending  the  conceptual  model  or 
conducting  additional  demonstrations  of  the  methodology  could  be  useful  and  complete 
projects.  Foi  example,  assessing  the  average  increases  in  time  required  for  planning 
caused  by  various  changes  in  the  ACMP  process  would  provide  ACC  a  baseline  for 
assessing  future  process  changes.  Given  the  critical  importance  unit-level  mission 
planning  played  in  Air  Force  success  in  Desert  Storm,  and  the  ongoing  organizational  and 
technological  trends  forcing  change,  additional  research  into  evaluating  the  ACMP 
process  will  continue  to  be  important  to  future  Air  Force  success. 
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ACMP  Process  Model 
Detailed  Backup  Material 


Overview 

The  information  contained  in  this  appendix  was  developed  to  augment  the  Air 
Combat  Mission  Planning  (ACMP)  process  model  developed  in  this  thesis.  For  that 
reason,  this  document  is  organized  along  the  lines  of  our  ACMP  model  process  phases 
and  functions.  While  not  formally  part  of  the  conceptual  model,  we  developed  this 
backup  material  as  a  companion  document  to  our  model.  This  appendix  further 
explains  the  ACMP  process  functions  by  providing  process  examples  and  function 
descriptions  that  we  developed  while  conducting  our  research. 

This  appendix  provides  our  account  of  how  we  see  the  ACMP  process  today, 
and  contains  information  that  is  more  specific  than  our  "generic"  mission  planning 
model.  Because  the  information  is  specific  to  today's  mission  planning  procedures,  it 
is  more  likely  to  become  outdated  as  the  process  undergoes  change.  Also,  because  the 
information  is  more  specific,  it  is  less  "generic"  and  admittedly  biased  toward  fighter 
air<to>ground  mission  planning.  Although  tliis  backup  material  was  reviewed  by  our 
mission  planning  experts  for  accuracy,  it  was  not  part  of  the  model  we  attempted  to 
validate  in  our  thesis.  Even  though  this  information  is  biased  and  less  "generic,"  it 
should  still  be  useful  to  individuals  searching  for  examples  and  farther  detail  than  is 
contained  in  our  ACMP  conceptual  model. 


A-1 


ACMLPrttg.fiS5 


Receipt  of  the  Air  Tasking  Order  (ATO)  begins  the  daily  cycle  of  mission 
planning  at  the  wing.  The  ATO  is  received  from  the  higher  headquarters  (HHQ)  and 
contains  the  tasking  information  for  all  units  involved  in  the  air  campaign.  Along  with 
specific  operational  information,  it  contains  the  miyor  high  level  constraints  that  must 
be  adhered  to  by  all  participants. 

The  ATO  assigns  all  aircraft  (strike  and  support)  involved  with  attacking  a 
particular  target  or  target  area  to  a  specific  mission,  designated  by  a  mission  number. 
Each  mission  is  assigned  to  a  Mission  Commander  who  is  an  experienced  pilot  and 
planner.  A  mission  may  involve  a  single  flight  of  similar  strike  aircraft  to  over  50 
individual  aircraft  of  various  types,  all  participating  in  different  strike  and  support 
roles.  The  ACMP  process,  which  begins  upon  receipt  of  the  next  day's  tasking,  is 
used  by  the  mission  commander  and  the  individual  flight  planners  to  plan  the  mission 
assigned  to  them  in  the  ATO. 

The  Mission  Planning  Phases 

For  discussion,  the  ACMP  process  is  broken  into  three  distinct  phases;  a  Before 
Flight  Planning  phase,  a  Flight  Planning  phase,  and  an  After  Flight  Planning  phase, 
Each  of  the  three  mission  planning  phases  is  broken  down  into  functions,  and  each 
function  will  be  fiirther  described  by  identifying  the  function's  objective  and  a  short 
narrative  about  how  the  function  is  normally  accomplished.  Four  events  are  key  to 
determining  the  timing  of  functions  within  the  ACMP  process;  1)  receipt  of  daily 
tasking  (e.g.  ATO),  which  defines  the  starting  point  of  the  process;  2)  the  beginning  of 
the  mass  briefing;  3)  the  beginning  of  the  flight  briefing;  and  4)  the  takeoff  time  for 
the  flight,  which  determines  the  end  of  the  ACMP  process  for  a  flight. 
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The  Before  Flight  Planning  Phase.  Planners  at  the  unit  level  carry  out  five 
functions  during  the  Before  Flight  Planning  phase.  The  model  refers  to  these 
functions  as:  Frag  Breakout,  Package  Planning,  Flight  Pre-Planning,  Mass  Brief,  and 
Structuring. 

Frag  Breakout.  The  objective  of  this  function  is  to  assess  the  ATO 
received  from  HHQ  to  determine  the  unit's  capability  to  accomplish  the  assigned 
missions,  and  allocate  specific  unit  resources  such  as  aircraft,  crews,  and  munitions  to 
those  missions. 

Currently  the  "Frag  Shop”  resides  at  the  wing  level  and  is  staffed  with 
representatives  from  maintenance,  munitions  (weapons),  and  operations  (flyers) 

(Bahnij,  1985:11-5).  The  ATO  (tasking)  is  broken  out  and  sorted  in  several  different 
formats  to  obtain  the  information  needed  by  the  unit's  organizations.  Coordination 
with  support  organizations  is  required  to  determine  if  an  adequate  number  of  aircrews, 
munitions,  and  aircraft  exist  to  meet  the  tasking.  If  sufficient  resources  do  not  exist  at 
the  unit,  the  staff  must  seek,  with  approval  of  the  Wing  Commander,  relief  from  HHQ. 
When  sufficient  resources  exist  at  the  wing,  the  resources  must  be  scheduled  and 
sorties  allocated  to  appropriate  squadrons.  Until  recently,  this  process  was  largely  a 
manual  or  semi-automated  process  that  required  a  minimum  of  4-6  hours.  Current 
efforts  are  underway  to  fully  automate  much  of  this  function  to  reduce  the  time 
required  to  30  minutes. 

Breaking  down  the  ATO  is  a  four-step  process.  First,  the  ATO  is  sorted  by  the 
unit  assigned  and  the  time  on  target  (TOT).  With  this  information  the  Frag  Shop  can 
put  together  a  rough  estimate  of  each  aircraft's  estimated  time  of  departure  (ETD). 
Aircraft  m.«untenance  shops  then  begin  estimating  their  work  load  and  ability  to 
support  the  next  day's  operations.  Similarly,  Operations,  Intelligence,  and  Weather 
support  personnel  estimate  their  support  requirements  and  begin  preparation  for  the 


next  day's  cycle.  Weapons  planners  check  the  list  of  munitions  assigned  tc  each 
target,  and  begif.  selection  of  weapons  for  those  not  assigned  by  the  HHQ. 

The  second  step  of  Frag  Breakout  involves  a  closer  look  at  the  ATO  sorted  by 
ETD  and  checks  it  against  the  base's  ability  to  launch,  recover,  and  return  aircraft  to 
service  given  the  estimated  flow  times.  Wing  priorities  are  developed,  and  a  more 
detailed  schedule  of  launch  times  is  established  nt  this  time.  With  this  additional 
information,  weapons  planners,  mission  planners,  and  mission  commanders  can  initiate 
aircrafVwe^ns  plaiming  in  the  Flight  Pre-Planning  and  Package  Plarniag  Functions. 

During  rhe  third  step,  crew  availability  is  paired  against  the  projected  wing 
schedule.  In  addition,  weather  forecasts  are  checked  to  determine  if  oiiy  weapon 
selections  should  be  changed.  The  Frag  Shop  planners  then  notify  the  wing 
commander  about  any  missions  that  cannot  be  supported  by  the  wing,  and  after 
receiving  die  wing  commander's  i^proval,  send  the  information  to  HHQ. 

The  fourth  step  of  the  Frag  Breakout  function  involves  finalizing  Mid 
publishing  the  wing's  flying  schedule  (the  Frag)  to  all  interested  organizations  in  the 
wing.  From  this  final  schedule,  all  support  organizations  can  finalize  their  support 
schedules,  squadrons  can  allocate  crews  to  sorties,  and  mission  planners  and  mission 
commanders  can  firm  up  their  plans. 

From  the  above  steps,  four  primary  decisions  to  be  made  can  be  identified  for 
this  function.  Tlie  staff  within  the  Frag  Shop  must  decide  which  aircraft  and  crews 
should  be  assigned  to  the  tasked  missions,  i^ich  munitions  should  be  used  against 
targets  that  have  not  been  "weapons  planned,"  which  missions  cannot  be  supported, 
and  finally,  what  is  the  next  day's  schedule  for  aircraft  and  aircrews 

To  make  these  decisions,  various  data  and  information  are  required.  Obviously, 
the  operational  information  contamed  in  the  ATO  is  mandatory.  Other  required 
operational  information  is  status  information  about  the  support  organization's  ability  to 
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generate  aircraft  or  to  provide  the  necessary  information.  Weather  forecasts  and 
intelligence  information  about  target  locations  and  defenses  must  be  provided  by  the 
intelligence  titid  weather  personnel.  In  addition  to  the  operational  information, 
weapons  performance  data  and  aeronautical  charts  are  required  to  select  the 
appropriate  munitions  and  to  estimate  distances  during  the  decision  process. 

The  final  product  from  this  function  is  a  flying  schedule  ar  1  mission  tasking 
(Frag)  for  each  sortie  to  be  flown  during  the  upcoming  squadron  dutj'  day. 

Information  contained  in  the  ATO,  now  combined  with  the  Frag,  is  passed  to  the 
planners  performing  concurrent  and/or  successive  functions  of  the  ACMP  process. 

Package  Planntny.  The  objective  of  the  Package  r*lanning  function  is  to 
develop  the  overall  tactics  for  the  packages,  coordinate  with  external  participants,  and 
prepare  a  "mass  brief  to  communicate  the  plan  to  unitdevel  participants  (Downer, 
1991:10).  Feedback  from  the  participants  about  mission  details  and  concerns  may 
occur.  This  function  is  accomplished  by  the  Mission  Commander  who,  "along  with 
other  fliglit  leaders,  starts  to  put  together  the  high  level  plan,  paying  particular 
attention  to  the  interfacing  issues,  coordinating  mutual  suppon  and  deconflicting  time 
and  space  requirsmen  {Bahnij,198S:  11-6).  If  tlie  mission  is  receiving  support  from 
squadrons  outside  the  wing,  flight  leaders  from  those  external  elements  participate  in 
the  coordination.  The  external  flight  leaders  are  responsible  for  completing  Before 
Flight  Planning  phase  actions  at  their  units. 

The  time  at  which  tliis  function  starts  varies  widely  depending  upon  the 
complexity  of  the  mission,  number  of  participants  invol  ved  in  the  mission,  and  the 
extent  of  support  provided  by  other  wings.  For  very  large  missions,  it  is  probable  that 
package  planning  may  begin  days  before  official  tasking  is  received  in  the  ATO.  For 
purposes  of  the  mode!,  it  is  assumed  that  the  bulk  of  the  package  planning,  as  it 
affects  the  rest  of  the  daily  mission  planning  process,  occurs  after  receipt  of  official 
tasking  on  the  day  prior  to  execution.  Given  that  assumption,  the  model  depicts 
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Package  Planning  starting  shortly  after  the  start  of  the  Frag  Breakout  function  upon 
availability  of  the  tasking  information.  Completion  of  this  function  is  driven  by  the 
need  to  conduct  the  mass  briefing  in  time  to  allow  all  participants  adequate  time  to 
complete  their  respective  flight  plans.  Inherently,  a  tradeoff  is  made  between  the 
amount  of  time  given  to  package  planning  and  the  amount  of  time  allocated  for 
individual  flight  planning. 

The  primary  decisions  made  by  the  mission  commander  or  planners  during  this 
function  are  what  tactical  constraints  (e.g.,  ROE,  sequencing)  should  be  imposed  on 
the  participating  flights,  what  is  the  best  way  to  use  the  resources  assigned  to  the 
mission,  at  >\^at  time  should  those  resources  be  used,  and  what  information  should  be 
presented  to  the  participants  at  the  mass  brief. 

Putting  together  the  tactical  plan  for  the  package  requires  extensive  data  and 
information.  The  package  plan  contains  such  elements  as  "a  package  flight  plan, 
launch  plan,  tanker  plan,  and  tactics  and  backup  plans"  (Downer,  1991:10).  In 
addition  to  the  tasking  information  contained  in  the  ATO  and  Frag,  the  Mission 
Commander  must  have  access  to  MCG&I  data,  parametric  data  on  various  aircraft, 
weapons,  threats  and  status  information  about  all  participants.  The  final  result  of  the 
mission  commander's  effort  is  the  package  plan  and  a  briefing  about  the  plan  that  is 
presented  during  the  mass  briefing. 

Flight  Pre-Planning.  Flight  pre-planning  is  performed  for  two  reasons; 

1 )  to  augment  and/or  reduce  the  flight  planning  required  by  crew  members  during  the 
duty  day;  and  2)  to  update  databases  and/or  planning  aids  prior  to  the  start  of  the  next 
24  hour  planning  cycle.  The  amount  of  Flight  Pre-Planning  conducted  by  a  wing  or 
squadron  varies  widely  depending  upon  many  factors.  These  factors  are  the 
wing/squadron  organization;  mission  type;  mission/package  size  or  complexity;  and  the 
degree  of  external  coordination  required  to  plan  the  mission. 
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Some  wings/squadrons  conduct  little  or  no  flight  planning,  leaving  it  all  for  the 
crews  to  accomplish  after  the  mass  brief.  Odier  units  employ  several  pilots  who  are 
not  on  flying  status,  or  intelligence  support  personnel,  to  maintain  planning  aids  (e  g., 
wall  charts,  threat  overlays)  for  use  by  mission  planners.  These  aids  are  usually 
constructed  prior  to  the  unit's  squadron  duty  day  and  maintained  throughout  the  cycle. 
More  recently,  an  increasing  number  of  units  have  begun  using  a  mission  planning  cell 
approach  to  augment  either  the  package  planner  >Aho  is  developing  the  package  plan, 
or  the  individual  crews  by  assisting  with  the  production  of  pre-planning  materials. 
These  pre-planning  materials  may  include  the  package  route  already  prepared  for  use 
in  individual  flight  plans,  specific  target  information,  photos,  printouts  of  important 
flight  plan  information,  or  many  odier  pieces  of  information  that  might  help  the  crews 
save  time  later  in  the  mission  planning  process  or  improve  t*'*)  quality  of  the  overall 
plan. 

This  activity,  if  it  occurs,  usually  takes  place  between  the  time  the  ATO/Frag 
arrives  at  the  unit  and  the  beginning  of  the  mass  brief  The  individuals  vdio  perform 
this  function  work  closely  with  the  mission  commanders  and  the  intelligence  personnel 
and  are  often  co-located  with  them.  The  decisions  made  during  this  function  are 
situational  and  determined  by  the  extent  of  flight  planning  conducted  during  this 
function. 

The  primary  products  of  this  function  are  the  pre-planning  materials  and  the 
planning  aids  which  will  be  used  by  the  aircrews  during  flight  planning. 

Mass  Brief  The  objective  of  the  mass  brief  is  to  disseminate  and 
clarify  the  package  plan  to  flight  planners  and  participants  prior  to  the  start  of  flight 
planning.  All  mission  participants  within  the  wing  or  squadron  usually  attend  the 
mass  brief  unless  currently  flying.  In  addition  to  mission  details  presented  by  the 
mission  commander,  a  general  weather  briefing  and  threat  briefing  is  given.  This 
status  information  helps  the  individual  flight  planners  decide  what  detailed  data  they 
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may  require  during  flight  planning.  A  detailed  list  of  topics  usually  covered  in  a 
mass  briefing  is  provided  in  Figure  A-1. 


Depending  upon  the  number  of  participants  and  number  of  missions  to  be 
flown  by  the  unit,  the  intelligence  and  weather  briefings  may  be  given  once  at  a 
central  location.  Each  group  assigned  to  different  packages  can  then  separate  and  be 
briefed  by  the  Mission  Commander  for  that  specific  mission,  lx  is  not  uncommon  for 
missions  scheduled  later  in  the  day  to  be  briefed  at  a  morning  mass  brief  This  is 
often  the  case  when  flying  combat  turns,  or  very  short  missioiis,  because  crew 
members  may  fly  several  missions  in  one  crew  day.  A  single  mass  brief  allows 
aircrews  to  spend  their  limited 
time  between  combat  missions 
preparing  flight  plans  and 
studying  the  target  area. 

Mass  brief  is  usually 
scheduled  to  occur  three  hours 
prior  to  package*launch  time 
(Downer,  1991:10).  Comments 
received  from  experienced  crew 
members  indicate  that  mass  brief 
usually  lasts  about  30  to  45 
minutes.  Few  major  planning 
decisions  are  made  during  this 
function  because  the  objective  is 
primarily  for  information 
dissemination.  Because  time  becomes  critical  as  takeoff  time  approaches,  any 
decisions  concerning  the  plan  now  are  usually  made  on  the  spot.  The  information 
requirements  of  this  function  are  primarily  contained  within  the  package  plan,  weatlier 
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'i,  ROU  CALL 

2.  TIME  HACK 

3.  MISSION  OBJECTIVBa 

4.  INTEL /THREAT 
8.  WEATHER 

e.  TAXI  /  MARSHALLING  /  ARMING  /  TAKEOFF  PLAN 

7.  RENDEZVOUS  PROCEDURES 

8.  TANKER  PUN 

e.  HOLDING  PROCEDURES  PRIOR  TO  JUMP  TIMES 

10.  INGRESS 

11 .  TARGETING  /  TOT  VMNDOWS  /  REATTACKS 

12.  EGRESS  FLOW  PUN 

13.  RENDEZVOUS  PROCEDURES 

14.  TANKER  PUN 

1B.  RECOVERY  /  SAFE  PASSAGE  PROCEDURES 
ie.  ECM  /  ECCM  /  COM  /  IFF  PROCEDURES 
17  8AR  PROCEDURES 
18.  SPECIAL  INSTRUCTIONS  /  RESTRICTIONS 
18.  ROE  (TRAINING) 

20.  QUESTIONS 

(H««Ma,  1980:11] 

U8AF  ineMbrWtipon*  RivMw 


Figuie  A>1:  Mass  Brief  Checklist 
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briefing,  and  intelligence  briefing.  Products  resulting  from  this  phase  are  the  package 
plan  and  flight  pre>planning  materials,  which  are  handed  out  just  prior  to  or  at  the 
mass  brief. 

Structuring.  The  objective  of  this  function  is  to  allocate  and  structure 
the  way  individual  flight  member(s)  will  perform,  assemble,  and  integrate  the  flight 
plan.  When  only  one  person  develops  the  flight  plan,  this  function  may  be  done 
mentally  as  the  planner  determines  the  approach  to  building  a  flight  plan.  As  more 
individuals  combine  their  efforts  to  build  the  flight  plan,  this  function  becomes  niore 
critical  to  achieving  a  quality  plan  in  an  efficient  manner. 

Flight  planning  tasks  may  be  divided  among  the  flight  members  to  capitalize  on 
the  team's  ability  to  accomplish  tasks  in  parallel.  This  is  a  common  practice  today  m 
many  units  when  planning  for  flights  of  multiple  aircraft  (Bahnij,  1985:11-10). 

Guidance  to  the  other  flight  members  may  need  to  be  issued  by  the  flight  lead  to 
establish  ground  rules  or  to  ensure  those  persons  building  separate  but  related  aspects 
of  the  flight  plan  (such  as  route  planning  and  target  area  tactics)  do  not  build  pieces  of 
the  plan  which  cannot  be  integrated. 

The  general  decisions  made  by  the  individual  performing  the  structuring  task 

are  these:  What  tactical  constraints  should  be  imposed  on  the  flight  plan? 

Who  should  perform  what  flight  planning  duties? 

In  what  order  should  the  steps  be  accomplished? 

What  guidance  should  be  provided  tn  the  flight  planners? 

The  data  required  to  accomplish  this  function  are  the  package  plan,  the  flight  pre¬ 
planning  materials,  and  operational  information  such  as  status  and  support  information, 
or  the  specific  tasking  contained  in  the  ATO.  The  product  of  this  function  is  the 
specific  guidance  given  to  the  flight  planners. 

The  Fllfht  Planning  Phase.  Seven  functions  comprise  the  Flight  Planning 
phase  of  the  ACMP  process.  They  are:  Data  Gathering,  Premission/TOLD,  Target 
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Study,  Weapons  Planning,  Route  Planning,  Target  Area  Tactics,  and  Avionics 
Preparation.  Each  of  these  functions  is  closely  tied  to  the  accomplishment  of  the 
other  functions,  and  there  is  no  clear  precedence  relationship  to  the  tasks.  For 
example,  some  individuals  prefer  to  begin  at  the  takeoff  point  and  plan  toward  the 
target,  while  others  prefer  to  begin  at  the  target  and  work  back.  In  fact,  flight  planners 
may  continually  jump  from  one  function  to  another  several  times  during  the 
construction  of  a  single  flight  plan,  sharing  information,  and  revising  pieces  of  the 
route  plan,  weapons  plan,  and  target  area  tactics  plan  based  on  ideas  or  plans  from 
another  function.  Several  of  the  functions  may  be  accomplished  in  parallel  by 
dividing  the  ftuictions  among  several  planners.  This  parallel  planning  approach 
increases  the  requirement  for  close  coordination  among  the  planners. 

Data  Gathering.  The  objective  of  Data  Gathering  is  to  collect  and  then 
coordinate  the  various  pieces  of  data/information  that  are  required  to  start  and  then 
conduct  individual  flight  planning  (e.g.,  Reconnaissance  Photos  and  Target  Area 
Forecasts).  The  amount  of  time  required  to  perform  this  function  varies  widely 
depending  on  numerous  factors  such  as  mission  type  and  weatlier.  A  miuor  portion  of 
this  function  occurs  at  the  beginning  of  the  Flight  Planning  phase  and  can  be  allocated 
to  a  specific  individual  of  a  multi-ship  flight.  Later  in  the  Flight  Planning  phuse,  this 
function  is  conducted  by  each  flight  planner  as  additional  information  is  required. 
Automation  can  greatly  assist  this  function  through  the  use  of  databases  and  computer 
query.  Decisions  made  by  the  flight  planners  performing  this  function  are; 

What  data  is  required? 

When  is  it  needed? 

Who  needs  it  (if  collecting  it  for  someone  else  in  the  flight)? 

Who  and  where  do  I  go  to  get  the  data? 

Premission/TOLD.  The  objective  of  the  premission/TOLD  function  is 
to  determine  the  initial  configuration  of  the  aircraft  and  then  calculate  the  Takeoff  and 
Landing  Data  (TOLD).  While  a  tedious  and  data  intensive  task  to  accomplish 
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manually  or  by  hand  held  calculator,  this  task  has  become  mostly  mechanical  with  the 
advent  of  computerized  mission  planning  systeips  and  parameter  databases. 

Information  and  data  collected  during  Data  Gathering  is  the  primary  source  of  input  to 
this  task.  The  information  generated  by  this  function  is  shared  among  the  other  flight 
planning  functions  such  as  route  planning,  avionics  preparation,  and  weapons  planning. 
In  addition  to  information  used  within  the  Flight  Planning  phase,  a  product  referred  to 
as  the  TOLD  piintout  is  frequently  generated  for  use  by  the  aircrew  as  part  of  the 
crew  material  taken  into  the  aircraft  and  used  during  the  mission. 

Route  Planning.  The  objective  of  Route  Planning  is  to  develop  the 
ingress/egress  route  profile  for  each  member  of  the  flight.  The  primary  decisions 
made  by  the  planner(s)  during  this  function  are;  1)  Where  to  fly  in  order  to  avoid  or 
minimize  exposure  to  the  threats,  and  2)  How  to  best  approach  the  target  area  to 
maximize  mission  success. 

The  distinction  between  this  function  and  the  Target  Area  Tactics  function 
involves  the  definition  of  "target  area,"  and  a  shift  in  emphasis  from  threat  avoidance 
as  the  overriding  factor  during  route  planning  to  the  emphasis  on  target  acquisition  and 
destruction  (or  other  mission  objective)  during  target  area  tactics.  This  shift  away 
from  threat  avoidance  is  not  meant  to  imply  that  threat  avoidance  is  not  important 
during  the  target  area  tactics  function,  only  that  relative  to  target  acquisition  and 
destruction,  it  is  of  secondary  importance. 

Accomplishing  this  function  requires  every  category  of  mission  planning  data. 
Inputs  are  needed  from  all  of  the  other  flight  planning  functions  and  almost  every 
other  function  requires  inputs  from  the  route  planning  function.  The  primary  output  of 
this  function  is  a  detailed  accounting  in  both  a  graphical  format  (strip  map)  and  tabular 
format  (e  g.,  AF  Form  70,  line-up  card)  of  exactly  where  the  pilot  intends  to  fly  the 
aircraft,  how  fast  he  intends  to  go,  what  altitude  he  plans  to  fly  at,  what  the  fuel  level 
at  each  route  point  should  be,  and  the  timing  between  points. 
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Target  Area  Tactics  Planning.  The  objective  of  Target  Area  Tactics 


Planning  is  to  develop  specific  tactics  for  each  flight  member  to  use  within  the  target 
area  that  attempts  to  maximize  mission  success,  minimize  danger  from  the  enemy,  and 
avoid  adverse  interaction  u4th  the  flight.  The  planner  is  concerned  with  what  general 
tactics  to  use,  how  the  timing  of  aircraft  attacks  should  be  orchestrated  to  avoid 
conflict  or  fratricide,  and  which  axis  of  attack  will  maximize  mission  success  and 
minimize  the  risk  to  the  aircrew.  For  instance,  the  flight  leader  may  decide  that  a 
flight  of  four  aircraft  will  split  into  two  groups  upon  entering  the  target  area.  The  first 
group  will  attack  from  the  south  while  the  second  group  comes  in  from  the  east.  The 
timing  is  planned  so  that  the  groups  attack  the  target  45  seconds  apart,  with  1 5  second 
separating  the  first  and  second  aircraft  in  a  group. 

These  types  of  decisions  would  be  based  upon  a  number  of  factors,  such  as  the 
threat  layout  around  a  target,  the  weapons  being  used,  the  number  of  aircraft  in  the 
target  area,  and  many  others.  A  detailed  study  of  the  target  tirea  (performed  in  the 
target  study  f\mction)  is  usually  conducted  prior  to  or  concurrent  with  the  target  area 
tactics  planning  function.  The  primary  result  of  this  function  is  a  target  area  tactics 
plan,  which  is  combined  with  the  route  plan  and  weapons  delivery  plan  to  create  an 
entire  flight  plan. 

Target  Study.  The  objective  of  Target  Study  is  to  gain  familiarity  with 
the  target  and  its  environment  relative  to  the  mission  tasking.  The  planner  may  study 
target  photos,  maps  of  the  target  area,  read  intelligence  reports,  or  generate  predictions 
of  radar  returns  in  order  to  improve  the  chances  that  the  target  can  be  quickly  acquired 
or  recognized  either  by  sight,  or  by  use  of  a  targeting  sensor  (e.g.,  radar,  infrared, 
electro-optical).  The  planner  may  note,  for  instance,  that  the  target  (say  a  building)  is 
the  third  one  on  the  left  from  a  large  steel  span  bridge.  Knowing  that  the  bridge  will 
produce  an  excellent  return  on  his  radar  from  a  much  greater  distance  than  he  could 
acquire  the  building  (either  on  radar  or  by  sight),  he  may  elect  to  navigate  and  target 
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(using  offset  aim  points)  using  the  bridge  instead  of  the  target.  By  studying  the  target 
tirea  in  detail  during  mission  planning,  the  planner  was  able  to  notice  and  possibly  use 
target  and  environmental  features  to  assist  with  mission  execution. 

In  addition  to  any  pre-planning  materials  or  planning  aids  tliat  may  have  been 
produced  prior  to  flight  planning,  the  planner  would  require  information  such  as  enemy 
intelligence  information,  or  target  photos  to  aid  the  familiarization  process.  Products 
resulting  from  this  phase  might  be  annotated  photos,  m^)8,  radar  predictions,  ;:)r  notes 
that  could  be  taken  into  the  cockpit  for  use  during  the  mission. 

Weapons  Planning.  The  objective  of  Weapons  Planning  is  to  determine 
the  best  employment  profile  to  use  given  route,  aircraft,  and  human  constraints.  Each 
weapon  system  has  optimal  methods  of  employment  against  different  types  of  targets, 
Accuracy  and  effects  are  determined  by  how  closely  the  aircrew  is  able  to  match  the 
actual  employment  profile  with  that  of  the  optimal  one  for  the  weapon,  Factors  make 
it  difficult  to  always  determine  the  optimal  method  of  employment,  even  if  the  optimal 
method  can  bo  used  given  other  considerations  such  as  exposure  time  to  certain 
threats,  obstructions,  or  high  chance  of  collateral  damage.  Weapons  Planning  is  the 
function  of  taking  these  factors  into  consideration  and  assessing  all  of  the  options.  In 
addition,  calculations,  such  as  whether  a  certain  pull  out  altitude,  from  a  certain  dive 
angle,  or  using  a  specific  bomb  and  fUse  combination  will  allow  the  aircraft  to  escape 
without  receiving  damage  from  its  own  weapons  delivery  must  be  assessed.  These 
types  of  issues  are  normally  addressed  during  the  Weapons  Planning  function. 

A  special  case  of  weapons  planning  involves  the  use  of  "smart"  weapons. 

These  systems  require  significant  planning  efforts  sometimes  equal  to,  or  exceeding, 
those  of  an  aircraft  flight  plan.  Guided  munitions  require  crew  planning  to  consider 
tactics  for  designating  the  target  or  maintaining  lock.  Autonomous  munitions  may 
require  the  crew  to  load  the  target  location,  navigation  information,  MCG&I  data  and 
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target  templates  for  use  by  the  munitions  censors.  All  of  tbiS  preparation  is 
considered  part  of  this  function. 

Avionics  Preparation.  The  objective  of  the  avionics  preparation 
function  is  to  specify  the  aircraft  a\'ionics,  weapons,  'ind  stores  configurations  needed 
for  initialization  of  the  circraft  computers.  With  the  arjvr  ,  ;  of  technology  leading  to 

■i  ’  i’  ■ 

even  more  computerization  of  our  aircraft  ar  J  117.1  amounts  of  data  about 

aircraft  and  weapons  configurations  are  requif  id  Ipy  airci  ilil^mputers.  Some  newer 

■ ;  '"i 

aircraft  can  already  accommodate  user  prefen  .leas  for  x4fault  display  options  or 
cockpit  display  layouts  which  are  initialized  by  the  pile.;  during  ground  operations.  In 
addition,  communications  settings,  navigation  points,  crypto  keying,  maps  and  terrain 
data,  and  other  avionics  settings  must  be  prepared  for  upload  to  the  aircraft  during  the 
mission  planning  process.  Tliis  step  of  the  ACMP  process  is  referred  to  as  the 
Avionics  Preparation  fimetion. 

Flight  planners  must  decide  what  aircraft  and  weapons  configurations  are 
necessary  given  the  specified  mission  and,  subsequently,  what  information  must  be  fed 
to  the  aircraft  to  support  that  configuration  choice.  Mission  information  such  as  the 
route  points,  communications  frequencies,  and  map  data  must  also  be  selected  and 
formatted  for  upload  to  the  aircraft.  How  difficult  this  avionics  preparation  task  is 
depends  upon  the  sophistication  of  the  aircraft's  systems  and  the  ease  with  which  the 
software  developed  to  aid  the  planner  interfaces  between  the  planner  and  tlie  aircraft 
computers, 

The  A^her  Flight  Planning  Phase.  The  four  functions  that  comprise  the  After 
Flight  Planning  Phase  of  the  ACMP  process  are  Flight  Plan  Integration,  Flight 
Briefing,  Ground  Operations,  and  Aircraft  Initialization.  These  functions,  unlike  the 
Flight  Planning  piiase,  generally  occur  in  sequential  order.  The  route  plan,  weapons 
delivery  plan,  and  the  target  area  tactics  plan,  which  aro  developed  within  the  Flight 
Planning  phase,  are  now  integrated,  checked  for  completeness  and  finalized.  The 
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individualii  of  tlie  flight  who  will  function  as  a  team  go  over  the  details  of  the  plan 
they  have  developed  and  ensure  everyone  has  an  understanding  of  how  it  will  be 
executed.  After  completing  this  flight  briefing,  the  aircrews  prepare  themselves  and 
their  aircraft  by  entering  or  uploading  the  mission  information  required  to  complete  the 
mission. 

Flight  Plan  Integration.  This  fiinction  is  similar  to  the  structuring 
function  in  that  it  may  vary  from  a  mental  process  in  the  case  of  a  singl''  planner,  to 
that  of  a  inqjor  integration  effort  in  which  the  various  flight  plan  pieces  are  developed 
by  different  members  of  the  flight.  Three  objectives  are  accomplished  during  the 
flight  plan  integration  function:  1)  the  individual  flight  plan  pieces  (route  plan, 
weapons  delivery  plan,  target  area  tactics  plan)  are  assembled  and  their  quality  c 
assured;  2)  the  crew  materials  (e.g.,  maps,  photos,  radar  predictions,  route  fonts)  are 
assembled,  finalized  or  generated;  and  3)  the  Data  Transfer  Device  (DTD),  if  needed, 
is  loaded  with  the  mission  information  required  to  initialize  the  aircraft  for  the 
mission. 

Data  requirements  for  the  function  involve  all  of  the  mission  information 
categories  associated  with  the  route  plan,  the  weapons  delivery  plan,  target  area  tactics 
plan,  and  avionics  configuration  settings  that  were  developed  during  the  Flight 
Planning  phase.  In  addition  to  these  mqjor  information  inputs,  the  planning  aids  and 
flight  pre-planning  materials  that  were  developed  during  the  Before  Flight  Planning 
phase  are  also  needed.  The  planner  is  concerned  with  how  well  the  individual 
elements  of  the  plan  fit  well  together  to  satisfy  the  mission  tasking,  and  whether  there 
is  any  mission  information  still  required  (threat  updates,  for  example). 

Flight  Briefing,  llie  objective  of  the  flight  briefing  is  to  review  the 
flight's  flight  plans  and  to  reinforce  the  important  points  and  procedures  necessary  to 
clarify  any  possible  ambiguities  which  may  exist  within  the  plan.  These  briefings  are 
generally  scheduled  so  that  upon  completion  of  the  flight  briefing,  an  hour  remains  for 
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the  crew  to  complete  ground  operations  and  aircraft  initialization  prior  to  takeoff 
Therefore,  the  completion  of  this  function  closes  the  window  of  opportunity  for  a 
planned  change  to  the  flight  plan,  because  the  remaining  time  until  takeoff  is 
consumed  by  activities  related  to  execution  preparation. 


PUQHT  BRIEPINQ 


INORE88 

I.  OTHER  PLAyeRS  /  RBNOBZVOUa  PROCBDURBS 
3.  ROUTE  OP  PLIOHT/ SPEED 

3.  THREAT  ARRAY  /  PBBA 

4.  FORMATION 

8.  ECM  /  BCCM  /  COMM  /  IFF  PROCEDURES 

6.  TIME  /  FUEL  CONTROLS  (OPS  OHECKS/BINOOS) 

7.  RADAR  VISUAL  SEARCH 

a.  THREAT  REACTIONS  VISUAL  /  RHAW  /  RADAR 

A,  SAMS /AAA 

B.  AIRCRAFT 
e.  UPDATE  POINTS 

10.  TERRAIN  CONSIDERATIONS 

I I .  TWO  /  THREE-SHIP  OPTIONS 

12.  DEQRADED  SYSTEMS 

13.  ROUTE  ABORT  PROCEDURES 

ATTACK 

1.  TERMINAL  THREATS  /  THREAT  REACTIONS 

2.  INITIAL  POINT 

3.  VRP/ VIP  DATA 

4.  IP  TO  TARGET 

A.  FORMATION /SPEED 

B.  TIME  /  HEADINO/  DiSTANCE 

C.  ACTION  POINT  /  TIMING  TO  POP  POINT 

8.  TACTICS 

A.  TYPE  OF  DELIVERY 


8.  TAROET 

A.  DESCRIPTION 

B.  WEATHER  /  SUN  ANCLE 

C.  VISUAL  CUB  /  BIC  TO  SMALL 

D.  SPECIFIC  AIM  POINT 

E.  RESTRICTIONS 

F.  TOT /REATTACK  WINDOWS 

7.  EORESB/OET  WELL  POINTS 

A.  REJOIN  PLAN 

B.  BOMB  /  BATTLE  DAMAGE  CHECK 

C.  TARGET  AREA  BINGO 
S.  CONTINOBNCieS 

A.  TWO  /  THREE-SHIP  OPTIONS 

B.  SYSTEM  OEGREDATION 

C.  ABORT  CODE  WORD 

D.  HUNG  /  UNEXPENDED  ORDNANCE 

E.  ASSYMETRIC  CONSIDERATIONS 
P,  JETTISON  PROCEDURES 

EGRESS 

1.  ROUTE  OF  FLIGHT /SPEED 

2.  THREAT  ARRAY  /  FEBA 

3.  FORMATION 

4.  SWING  PROCEDURES 

8.  ECM  /  EOCM  /  COMM  / 

IFF  PROCEDURES 

8.  TIME  /  FUEL  CONTROL 


B.  SWITCHOLOGY  /  SMB  SETTINGS 
C  PARAMBTBN8 

D.  TYPE  OF  SAFE  SEPARATION  FOR  WINGMEN 

E.  REATTACK  /  WEATHER  BACK-UP  PLAN 


(OPSCHECKS/BINOOS) 

7.  TERRAIN  CONSIDERATIONS 

[Hatittt,  1890:13] 

USAF  PIflhtir  Wiipont  R«vl«w 


Figure  A-2;  Flight  Briefing  Checklist 

In  addition  to  reviewing  the  integrated  flight  plan  and  operational  procedures, 
last  minute  updates  to  information  on  enemy  intelligence,  weather,  tasking  and  support 
are  discussed  and  crew  materials  are  annotated  with  any  last  minute  notes  from  the 
flight  briefing.  A  detailed  list  of  the  items  covered  in  the  flight  briefing  is  provided  in 
Figure  A-?,. 
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Ground  Operations.  Ground  Operations  includes  all  those  activities 
which  must  be  accomplished  by  the  pilot  aod/or  crew  from  the  end  of  the  flight 
briefing  until  the  aircraft  is  ready  to  be  initialized  with  the  mission  information. 

These  activities  may  include  suiting  up  prior  to  the  flight,  obtaining  last  minute 
intelligence  and  weather  briefings,  filing  flight  plans,  pre*flighting  the  aircraft, 
performing  an  aircraft  or  weapons  safety  inspection,  and  starting  aircraft  engines. 

While  this  function  does  not  specifically  contain  planning  activities,  it  is  important  to 
the  ACMP  process  because  it  affects  the  time  necessary  to  mission  plan.  Crew 
members  normally  anticipate  that  these  ground  operations  will  require  about  one  hour 
to  complete.  Therefore,  when  considering  the  amount  of  time  available  to  develop  the 
flight  plans,  the  aircrews  usually  reserve  one  hour  to  perform  this  function. 

Aircraft  Initialization.  The  objective  of  this  function  is  to  enter  or 
upload  all  mission  specific  data  into  the  aircraft/weapon  computers  prior  to  take-off. 
This  function  is  usually  automated  by  the  use  of  a  Data  Transfer  Device  (DTD).  The 
DTD  is  a  removable  memory  device  which  can  be  loaded  from  a  mission  planning 
computer  that  is  available  in  the  squadron.  This  device  is  carried  by  the  pilot  to  the 
jet  and  plugged  into  a  specialized  port.  All  of  the  information  that  was  loaded  into  the 
DTD  during  the  flight  planning  functions  is  loaded  into  the  aircraft  or  weapon 
computers.  In  the  event  that  the  aircraft  does  not  have  the  capability  to  receive  data 
through  a  DTD,  or  if  the  DTD  is  not  functioning,  the  pilot  can  manually  enter  the 
information  in  the  cockpit.  This  manual  method  is  extremely  slow  and  prone  to  error, 
information  such  as  threat  locations/types,  navigation  points,  route  depictions, 
communication  frequencies,  weapons  parameters,  target  locations  and  even  map/photo 
information  can  be  loaded  if  the  capability  is  provided  in  the  aircraft/weapon. 

After  the  aircraft  initialization  information  has  been  loaded  (manually  or 
through  a  DTD)  and  verified,  the  pilot  and  aircraft  are  ready  to  proceed  with  takeoff 


A-17 


and  the  mission.  Once  the  aircraft  has  been  initiali-^ed  with  the  mission  information, 
the  ACMP  process  is  ended  for  that  flight  or  sortie. 

Mission  Planning  Data  Categories 


In  this  conceptual  model,  categories  of  mission  planning  data  have  been 
suggested.  The  five  level  1  categories  have  been  adopted  from  the  .A.ir  Force  Concept 
of  Operations  Document  for  the  Air  Force  Mission  Support  System  fAFMf  SI.  dated  8 
November  1991.  The  following  sections  discuss  these  level  1  categories.  More 
detailed  information  on  these  categories  and  their  level  2  categories  is  provided  at 
Appendix  B.  Also  at  Appendix  B  are  examples  of  information/data  which  would  be 
included  at  level  3  to  help  the  reader  understand  each  category. 


Parametric  Data.  Parametric  data  defines  aircraft  and  weapons  performance 
and  signature  mod'*’'!.  The  original  source  for  this  information  is  the  aircraft/weapon 
manufacturer  or  designated  MAJCOM  agencies.  This  data  is  the  basis  for  fliuht 
planning  of  fuel,  distance,  and  time  calculations.  Parametric  data  is  relatii'el>  j.;atic 
and  usually  will  be  incorporated  into  software  or  databases. 

MCG&I  Data.  MCG&I  data  includes  digital  maps,  charts,  terrain  elevation, 
cultural  features,  imagery,  and  other  types  of  aeronautical  information.  The  origin  for 
most  of  this  data  is  the  Defense  Mapping  Agency  (DMA).  The  map  data  provides  the 
primary  background  for  mission  planning  and  all  associated  threat  and  informational 
displays.  MCG&I  data  is  also  relatively  static,  changing  in  response  to  infrequent, 
generally  planned  update  procedures. 

Operational  Information.  Operational  information  is  input  data  relevant  to  the 
unit's  area  of  intcrert,  including  intelligence  data  (e.g.,  enemy  threats),  operations  data 
(e.g..  ATO),  and  weather  data.  This  data  is  the  basis  for  planning  optimum  ingress 
and  egress  routes  and  mission  accomplishment.  Operational  information  is  expected  to 
change  frequently  and  unpredictably,  and  will  therefore  be  provided  by  electronic  data 
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feeds  when  and  where  available,  by  removable  secondary  storage  media  as  a  fall  back 
option,  or  when  security  constraints  dictate,  and  by  manual  input  when  other  solutions 
are  not  available. 

Mission  Information.  Mission  information  is  the  result  of  the  mission  planning 
process  and  the  mission  itself.  The  user  interacts  with  the  system,  its  resident 
algorithms,  and  operational,  MCG&I,  and  parametric  databases,  to  produce  the  CMF 
and  load  a  DTD.  The  DTD  contains  aircraft/weapons  navigation  and  avionics 
initialization  data.  Mission  information  also  includes  those  intermediate  plans  which 
comprise  an  integrated  flight  plan. 

Setup  Information.  Setup  information  includes  unit/user  unique  data  and  the 
way  point  library.  The  unit^user  data  includes  security  data,  and  profiles  for  default 
settings,  mission  planning,  and  forms/CMF  generation.  This  data  is  usually  entered 
manually.  Setup  information  is  expected  to  change  slowly  under  control  of  the  user. 
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Appendix  B;  Level  2  Data  Categories  and  Examples 


Overview 

Our  conceptual  model  specifies  and  defines  five  level- 1  categories  of  mission 
planning  data;  1)  parametric  data;  2)  mapping,  charting,  geodesy,  and  imagery 
(MCG&I);  3)  operational  information;  4)  mission  infoimation;  and  5)  setup 
information.  These  five  level- 1  categories  were  adopted  from  the  Air  Force  Concept 
QLQD.erations  Document  for  the  Air  Force  Mission  Support  System  (AFMSSl.  dated  8 
November  1991.  We  then  subdivided  the  five  level- 1  categories  into  twenty  level-2 
categories  in  order  to  provide  more  detail  and  insight  into  the  information  requirements 
of  mission  planners. 

In  this  appendix  we  attempt  to  further  define  the  ACMP  process  data 
requirements  by  looking  at  some  examples  in  each  of  the  level-2  categories.  We  also 
did  this  to  try  and  help  readers  understand  what  types  of  information  might  be 
associated  with  each  of  the  level-2  categories.  We  do  not  suggest  that  these  examples 
comprise  a  complete  list  of  mission  planning  data  found  at  the  next  lower  level  of 
data,  or  that  they  necessarily  represent  good  candidates  for  category  labels  at  level-3. 
Each  of  the  following  pages  of  this  appendix  lists  a  level-l  data  category  at  the  top  of 
the  page.  Below  each  level- 1  category  are  listed  its  level-2  data  categories  followed 
by  some  examples  of  each  level-2  category. 
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Paramytrit  Pata 

Aircraft  Performance  Data 
examples; 

Fuel  Flow  Rates 

Weight 

Drag 

Radar  Cross  Section 

Turning  Radius 

Size 

Climb  rates 

Range 

Speed 

etc... 

* 

Weapons  Performance  Data 

examples: 

Drag 

Range 

Lock -on  Distance 

Accuracy 

Frag  Pattern 

Fuzing  Options 

Speed 

Weight 

JMEM 

Threat  Performance  Data 
examples; 

Missile  Range 

Radar  Detection  Range 

Frequencies 

Antennae  Height 

Radar  Modes 

Name/variation 

Max  Altitude 

Minimum  Altitude 

• 
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MCG&l  fMapping.  Charting.  Geodesy,  and  Imagery) 


Aeronautical  Maps  and  Charts 
examples; 

Paper  Charts  (1:50,  TPC,  JOG.  ONC) 

Digital  Maps 
CHUM 

Esct4)e  and  Evasion  Charts 
Terrain  Elevation  Data 

examples:  Digital  Terrain  Elevation  Data  (DIED) 

Cultural  Feature  Data 

examples:  Digital  Feature  Analysis  Data  (DFAD) 

Digital  Airfield  Information  File  (DAFIF) 
Vertical  Obstruction  Data 

Imagery 

examples: 

National  High  Altitude  Photography  (NHAP) 

LANDSAT 

SPOT 

Reconnaissance  Photos 
BTG's  (Basic  Target  Graphics) 

Radar  Imagery 
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Operational  Information 


Enemy  Intelligence  Information 

examples: 

Threat  Locations  Enemy  Intentions 

Enemy  Movement  Enemy  Strength 

Target  Location  Target  Composition 

Enemy  Tactics/Procedures 
Enemy  Weapon  System  Capabilities/Shortfalls 


Friendly  Intelligence  Information 
examples; 

Friendly  Force  Locations 
Allied  Movement 

Weather  Information 

examples: 

Target  Area  Forecast 

Winds 

Humidity 

Frontal  Movements 
Transmissitivity 
Sun/Moon  angles 

Tasking  Information 
examples; 

TOT 

Objective 
Munitions 
unit(s)  assigned 
call  signs 

Status/Support  Information 
examples; 

Munitions  availability 
Aircrew  Status 
Support  assets  assigned 
LLTR's 
Tanker  Orbits 
Operating  Frequencies 


Allied  Intentions 
Allied  Tactics/Procedures 


Visibility 
Temperatures 
Cloud  Levels 
Precipitation  Levels 
Solar/Lunar  timing 
Tides 


Mission  Number 

Rules  of  Engagement  (ROE) 

DMPI 's 

aircraft  type 

Mission  Commander  Assignments 


Aircraft  Status 
Base  Status 

Restricted  Operating  Zones 
Free  Fire  Zones 
SAR  Procedures 
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Mission  Information 


Navigation  Information 
examples; 

TOLD 
Way  Points 
Divert  Points 
Time  enroute 
Takeoff  Time 
Fuel  Levels 
Airfield  Locations 
ROZ's 


Turn  points 

IP,  GAP'S  VRP's,  VIP's 

Headings 

Elapsed  Times 

Airspeed 

Distance 

LLTR's 


Route/Target  Depictions 

examples; 

Strip  charts 

Target  Photos 

Scene  Perspective  Views 

Communication  Settings 
example; 

Radio  Frequencies 
TACAN  Frequencies 

Weapons  Release  Settings 

examples; 

Fuzing 

Release  Altitude 
Aimpoint 


Radar  Predictions 
Targeting  Templates 


Call  Sign  Lists 


Dive  Angle 
Release  Airspeed 


Avionics  Settings 
examples; 

Crypto  Keying  GPS  Effemeral  Data 

Cockpit  Display  Configuration  INS  Check  points  (swing  checks) 
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Defaults 

examples; 

Airfield  Locations  Aircraft  Weights 

SCL's  Security  Passwords 

Form  formats  Canned  communication  files 


Point  Libraries 
examples 

Pre-stored  waypoints  Standard  Departure  Routes 

Standard  Approach  Routes 
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Appendix  C:  ACMP  Process  Function  Summary  Sheets 


Overview 

The  ACMP  process  fimction  summary  sheets  found  in  this  appendix  are  the 
result  of  our  attempt  to  consolidate  the  information  contained  in  the  ACMP  model 
gr^hics  (Figures  4*1  through  4>12B)  into  a  concise  written  format.  These  summary 
sheets  were  developed  from  the  working  papers  we  used  to  organize  our  thoughts  and 
build  the  ACMP  conceptual  model.  We  believe  they  complement  the  conceptual 
model  by  providing  information  about  each  specific  ACMP  process  function  on  one  or 
two  sheets,  as  opposed  to  being  shown  on  several  figures  in  the  thesis  figures. 

Each  sheet  provides  the  function  name;  the  function  objective;  a  list  of  primary 
decisions  made  by  plarmers  performing  the  function;  the  level*!  and  level*2  data 
requirements  of  the  function;  the  name  of  the  previous  and  successive  function 
performed  by  the  plarmer;  products  produced  by  planners  performing  the  current 
function;  and  finally,  special  notes  about  the  function. 
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Function  Name:  Frag  Breakout 


Objective:  Assess  the  tasking  received  from  higher  headquarters  (HHQ)  to 
determine  the  units  capability  to  accomplish  the  assigned 
missions,  and  allocate  specific  unit  resources  such  as  aircraft, 
crews,  and  munitions  to  those  missions. 

Decisionfs):  •  Which  squadrons  should  be  assigned  to  the  tasked  missions. 

-  Which  munition  should  be  used  against  the  assigned  targets. 

•  Which  missions  to  return  to  higher  headquarters. 

*  What  is  the  schedule  for  aircraft  and  squadrons. 

Data  Required: 

-  Parametric 

-  Weapons  Performance  Data 

-  MCG&I 

-  Aeronautical  Maps  and  Charts 

-  Operational  Information 

“  Tasking  Information 

-  Status/Support  Information 
--  Enemy  Information 

-  Weather 

Previous  Function:  Receipt  of  Tasking  (External  to  Model) 

Successive  Functions:  Package  Planning  and  Flight  Pre-Planning 
ProductfsV  -  Frag 


Notes:  The  model  assumes  that  the  majority  of  tasking  is  recieved  from 

the  Air  Tasking  Order  (ATO)  every  24-hours.  The  ATO  provides 
the  data  required  to  start,  and  retasking  is  a  change  or  addition  to 
a  previous  ATO. 
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Function  Name:  Package  Planning 


Objective:  To  develop  overall  tactics  for  the  package,  coordinate  with 

external  participants,  and  prepare  a  "mass  brief  to  communicate 
the  plan  to  unit-level  participants. 

Decislonfs):  -  What  tactical  constraints  to  impose  on  flights  (e.g.,  ROE's). 

-  What  external  resources  to  use  and  when. 

-  What  information  to  present  during  the  mass  briefing. 

!2flia.,RgqMirp^; 

-  Frag 

‘  Parametric  Information 

“  Aircraft  Performance  Information 

-  Weapons  Performance  Information 

-  Threat  Performance  Information 

-  MCG&I 

-  Aeronautical  Maps  and  Charts 
~  Terrain  Elevation  Data 

-  Cultural  Feature  Data 

-  Imagery 


-  Operational  Information 

-  Enemy  Intelligence  Information 

-  Friendly  Intelligence  Information 

-  Weather  Information 

-  Tasking  Information 

“  Status/Support  Information 

Previous  Function:  Receipt  of  Tasking  (External  to  Model) 
Successive  Function;  Ui:m  ^rief 

Productfis;:  -  Package  Plan 

-  Briefing  Outwne 


Notoc:  The  magnitude  of  this  effort  is  dependent  upon  the  size  and  type 

of  mission,  the  number  of  packages,  and  the  planning  latitude 
given  to  the  mission  commander  by  higher  headquarters. 
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Function  Name:  Flight  Pre-Planning 


Objective:  Flight  pre-pianning  ha«  two  related  objectives;  1)  accomplish 
preliminary  flight  planning  to  augment  and/or  reduce  the  flight 
planning  required  by  crew  members  during  the  duty  day,  and  2) 
update  databases  and/or  planning  aids  (wall  charts,  overlays,  etc.) 
prior  to  start  of  the  next  24  hour  planning  cycle. 


Decislon/sV  The  decisions  are  situationally  determined  by  the  extent  of  flight 

planning  conducted  during  this  function.  Refer  to  the  specific  flight 
planning  functions  for  further  detail.  The  extent  of  flight  planning 
conducted  during  this  function  is  determined  by  the  wIng/squadron 
organization,  mission  type,  mission/package  sizes,  and  degree  of 
external  coordination  required. 

Data  Required: 

-  Package  Plan 

-  Frag 

•  Parametric  Data 

-  Aircraft  Performance  Data 

“  Weapons  Perfomiance  Data 
“  Threat  Performance  Data 

•  Operational  Information 

-  Enemy  Intelligence 
--  Friendly  Intelligence 
--  Weather 

-  Tasking  Information 

--  Status/Support  Information 

-  MCG&I 

-  Aeronautical  Maps  and  Charts 
--  Terrain  Elevation  Data 

-  Cultural  Feature  Data 

-  Imagery 

Previous  Function:  None.  Receipt  of  Tasking  (External  to  Model) 

SiJggSSSiYJg  iMn.Ctigp:  Mass  Brief 
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Productrs):  -  Flight  Pre-Planning  Materials  (e.g.,  Form  70s,  Data  Transfer 

Devices  (DTD),  Radar  Predictions,  Target  Photos) 

-  Planning  Aids 

Notes:  The  magnitude  of  this  function  ranges  from  non-existent  to  a 

multiday  effort.  It  focuses  on  the  intelligence  and  design  phases  of  decision 
making. 
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Function  Name:  Mass  Brief 


Objective:  To  disseminate  and  clarify  the  package  plan  (objective, 

constraints,  procedures,  etc.)  to  all  flight  planners  and  participants 
prior  to  the  start  of  flight  planning.  Aaditionally,  the  mission 
commander  receives  feedback  on  package  plans  and  may  make 
spot  adjustments. 

Decisionfs^:  -  What  (if  anything)  in  the  package  plan  is  unacceptable. 

-  What  are  external  factors  affecting  mission  execution. 

Data  Required: 

-  Package  Plan 

-  Operational  Information 

-  Weather  (briefing) 

-  Enemy  Intelligence  (briefing) 

-  Friendly  Intelligence  (briefing) 


Previous  FunctlonfsV  Package  Planning  and  Flight  Pre-Planning 

Successive  Function:  Structuring 

Product^sl:  -  Dissemination  of  Package  Plan 

-  Dissemination  of  Flight  Pre-Planning  Materials 

Notes:  Mass  brief  is  routinely  scheduled  to  begin  three  hours  prior  to  the 

first  takeoff.  Its  start  opens  the  window  of  time  available  for  crew 
planning,  (see  briefing  outline/checklist).  Multiple  missions  may 
be  briefed  at  a  single  mass  briefing,  with  Intelligence  and  Weather 
briefings  being  attended  by  all  participants.  After  reciept  of  the 
Intelligence/Weather  briefings,  mission  commanders  may  separate 
to  brief  the  individual  missions. 


C-6 


Function  Name:  Structuring 


Objective:  To  allocate  and  structure  how  the  individual  flight  member(s)  will 
perform,  assemble,  and  integrate  the  flight  plan. 

Pedsionfs):  -  What  tactical  constraints  to  impose  on  the  flight  plan 

-  Who  should  perfonn  what  flight  planning  steps 

-  In  what  order  should  the  steps  be  accomplished 

-  What  guidance  to  provide  for  flight  planning 

Data  Required: 

-  Package  plan 

-  Flight  Pre-Planning  Materials 

-  Operational  Information 

-  Status/Support  information 

-  Tasking  Information 


Previous  Function:  Mass  Brief 
Successive  Function:  Flight  Planning 


ProductfsV  Planning  Guidance  to  flight  members  (e.g  ,  define  target  area, 
assign  flight  planning  function  responsibilities). 


Notes:  Flight  may  consist  of  one  person  who  self-structures  or  several 

persons  with  a  designated  flight  leader.  The  successive  “function" 
of  Flight  Planning  incorporates  several  functions  which  do  not 
have  precedence  relationships. 
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Function  Name:  Data  Gathering  (Flight  Planning) 


Objective:  To  collect  and  coordinate  various  data/information  required  to 
begin  and  then  conduct  individual  flight  planning. 

Decisionfs):  -  What  data  is  required? 

-  When  is  it  needed? 

-  Who  needs  it? 

-  Who  and  where  to  get  the  data  from. 

Data  Required: 

-  Package  Plan 

-  Planning  Aids 

-  Planning  Guidance  (from  structuring  phase) 

-  Flight  Pre-Planning  Material 

-  Operational  Information 

•  Status/Support  information 
-  Tasking  Information 

-  information  on  data  availability 
Previous  Function:  Structuring 
Successive  Function-  Flight  Plan  Integration 

ProducttsV  -  Data  required  for  all  other  Flight  Planning  functions. 

Notes: 
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Function  Name:  Premission  TOLD  (Flight  Planning) 

Objective:  To  determine  aircraft  Take-Off  and  Landing  Data  (TOLD). 

Decision(s):  -  What  is  aircraft  takeoff  weight? 

-  What  is  aircraft  takeoff  distance? 

•  What  is  aircraft  takeoff  speed? 

•  What  is  aircraft  emergency  landing  weight? 

-  What  is  aircraft  emergency  landing  distance? 

-  What  is  aircraft  emergency  landing  speed? 

Data  Required: 

-  Setup  Information 

-  Defaults 

-  Parametric  Data 

-  Aircraft  Performance  Data 

•  MCG&I 

-  Cultural  Feature  Data 

-  Operational  Information 

-  Tasking  information 

-  Weather  Information 


Previous  Function:  Structuring 

Successive  Function:  Flight  Plan  Integration 

ProductfsV  -  Mission  Information 

-  Navigation  information  (TOLD  printout) 

Notes: 
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Function  Name:  Route  Planning  (Flight  Planning) 

Objective:  To  develop  the  ingress/egress  rou+.e  for  the  flight 

Decision(s):  -  Where  to  fly  to  avoid  the  threat 

-  How  to  best  approach  the  target  area  to  maximize  mission 

success. 

Data  Required: 

-  Package  plan 

-  Planning  Guidance 

-  Flight  Pre-Planning  Materials 

-  Planning  Aids 

-  Parametric  Data 

-  Aircraft  Performance  Data 

~  Weapons  Performance  Data 
~  Threat  Performance  Data 

-  Operational  Information 

-  Enemy  Intelligence 

-  Friendly  Intelligence 

-  Weather  Information 
~  Tasking  Information 

-  Status/Support  Information 

-  MCG&I 

-  Aeronautical  Maps  and  Charts 
••  Terrain  Elevation  Data 

-  Cultural  Feature  Data 

-  Imagery 

-  Setup  Information 

~  Defaults 
*-  Point  Librarys 

Previous  Function:  Structuring 

Successive  Function:  Flight  Plan  Integration 
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Route  Plan 


Notes:  The  ingress  route  ends  at  the  boundary  of  the  target  area.  The 

egress  route  begins  either  at  the  target  or  boundary  of  the  target 
area  depending  on  the  unit's  convention. 
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Function  Name:  Target  Area  Tactics  (Flight  Planning) 

Objective:  Develop  specific  tactics  for  each  flight  member  to  use  within  the 

target  area  which  attempts  to  maximize  mission  success,  minimize 
danger  from  the  enemy,  and  avoid  adverse  interaction  within  the 
flight. 

Decisionfsl:  -  What  general  tactics  to  employ  within  the  target  area. 

-  What  is  the  specific  timing  and  flight  path  for  each  flight  member. 

Data  Required: 

•  Package  Plan 

-  Planning  Guidance 

-  Flight  Pre-Planning  Materials 

-  Planning  Aids 

-  Parametric  Data 

-  Aircraft  Performance  Data 

-  Weapons  Performance  Data 

-  Threat  Performance  Data 

•  Operational  Information 

-  Status/Support  Information 

-  Tasking  Information 

--  Enemy  Intelligence  Information 

-  Weather  Information 

-  MCG&l 

"  Aeronautical  Maps  and  Charts 

-  Terrain  Elevation  Data 

-  Cultural  Feature  Data 
“  Irhagery 

-  Mission  Information 

-  Navigation  Information 

-  Route/Target  Depictions 

Previous  Function:  Structuring 
Successive  Function:  Flight  Plan  Integration 
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ProductrsV  -  Target  Area  Tactics  Plan 

Notes:  Generally,  some  Target  Study  is  required  in  conjunction  with  this 
function. 
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Function  Name:  Target  Study  (Flight  Planning) 


Qbiective:  To  gain  familiarity  v/ith  target  and  environment  relative  to  the 
tasking. 


DecisionfsV  What  target  and  environment  features  can  assist  in  mission 
execution  (e.g.,  off-set  aim  points). 


Data.  Required: 

-  Right  Pre-Planning  Materials 

-  Planning  Aids 

-  Target  Area  Tactics  Plans 

-  MCG&i 

-  Aeronautical  Maps  and  Charts 

-  Terrain  Elevation  Data 

-  Cultural  Feature  Data 

-  Imagery 

•  Operational  Data 

-  Enemy  Intelligence  Data 

-  Friendly  Intelligence  Data 

-  Weather  Information 
“  Tasking  Information 

“  Status/Support  Information 

-  Mission  information 

"  Navigation  information 

-  Route/Target  Depictions 

Previous  Function:  Structuring 
Successive  Function:  Flight  Plan  Integration 

ProductfsV  -  Mission  Information 

-  Routerr arget  Depictions 

-•  Characteristics  and  features  to  exploit  during  execution 

Notes:  There  are  two  primary  phases  of  target  study.  The  first  is  conducted  in 
conjunction  with  target  area  tactics  development  by  persons  responsible 
for  target  area  tactics.  During  the  second  phase  all  flight  members 
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conduct  target  study  relative  to  the  tactics  each  is  assigned  by  the 
target  area  tactics  and  weapons  planning  functions. 
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Function  Name:  Avionics  Preparation  (Flight  Planning) 

ObiectiveTo  specify  aircraft  avionics,  weapons,  and  stores  configurations. 
Decision(s):-  Given  the  mission,  what  configurations  are  appropriate? 

Data  Required: 

•  Parametric  Data 

-  Aircraft  Performance  Data 

-  Weapons  Performance  Data 
Threat  Performance  Data 

•  MCG&I 

-  Aeronautical  Maps  and  Charts 
«  Terrain  Elevation  Data 

Cultural  Feature  Data 
--  Imagery 

-  Mission  Information 

-  Navigation  Information 

-  Communication  Settings 
Weapons  Release  Settings 

-  Avionics  Settings 

*  Setup  Information 

..  Defaults 

-  Point  Librarys 

Previous  Function:  Structuring 
Successive  Function:  Flight  Plan  Integration 
Productfs):  -  Avionics  Configuration  Settings 

Notes:  Aircraft  require  initial  entry  of  data  into  avionics.  Generally, 

aircraft  systems  provide  specific  software  which  accepts  initial 
values  from  the  general  mission  planning  software  and/or  the 
aircrew.  The  relative  sophistication  of  the  interface  between 
airaaft  specific  and  general  mission  planning  software  influences 
how  units  conduct  the  mission  planning  process. 
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Function  Name:  Weapons  Planning  (Flight  Planning) 


Objective:  To  develop  employment  profile  to  maximize  weapon  effectiveness 
and  crew  safety. 

DedsionfsV  -  For  each  target  run,  which  delivery  profile  and  weapon  settings 
maximize  mission  success  and  minimize  risk  to  the  aircrew? 

-  For  "smart"  munitions,  what  sensor  data  should  be  loaded  into 
the  weapon? 

•  Package  Plan 

-  Planning  Guidance 

-  Flight  Pre-Planning  Materials 

-  Planning  Aids 

-  Parametric  Data 

-  Weapons  Performance  Data 

•  MCG&! 

••  Aeronautical  Maps  and  Charts 

-  Terrain  Elevation  Data 

-  Cultural  Feature  Data 
“  Imagery 

-  Operational  Information 

-  Enemy  Intelligence  Information 

-  Weather  Information 
~  Tasking  Information 

-  Status/Support  Information 

•  Setup  Information 

..  Defaults 

•  Mission  Information 

-  Navigation  Information 

-  Route/Target  Depictions 

-  Weapons  Release  Settings 

Previous  Function:  Structuring 
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Successive  Function:  Flight  Plan  Integration 


ProducttsV  -  Weapons  considerations  for  target  area  tactics 
.  Weapons  Delivery  Plan  (and  data  file  if  required) 


Notes:  Aii  weapon/aircraft  configurations  have  limitations  on  their 

employment  to  ensure  safety  and  effectiveness  (e.g.,  release 
altitudes,  speeds,  and  weather  constraints).  Additionally,  smart 
munitions  require  special  planning.  Guided  munitions  require  crew 
planning  to  consider  tactics  for  designating  the  target. 

Autonomous  munitions  require  the  crew  load  the  target  (and 
potentially  route)  data  required  by  the  munition's  sensors. 
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Function  Name:  Flight  Plan  Integration 

Objective:  -  To  assemble  the  individual  flight  plan  pieces  and  assure  the 
quality  of  the  integrated  plan. 

-  To  generate  and  assemble  the  crew  materials  into  a  Combat 
Mission  Folder 

-  To  load  a  Data  Transfer  Device  (DTD) 

Decisionfs^:  -  Do  the  elements  of  the  plan  fit  together  to  satisfy  the  tasking? 

-  Is  any  mission  information  still  required? 

Data  Required: 

-  Package  Plan 

-  Flight  Pre-Planning  Materials 

-  Planning  Aids 

-  Route  Plan 

-  Target  Area  Tactics  Plan 

-  Weapons  Delivery  Plan 

-  Avionics  Configuration  Settings 
Previous  Function:  Flight  Planning 
Successive  Function:  Finalize  Crew  Materials 

ProductlsV  -  Integrated  Flight  Plan 

-  Loaded  DTD 

-  Crew  Material 

Notes:  Individual  mission  data/information  categories  are  subsumed  within  the 
individual  plans  at  this  point  of  the  process. 
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Function  Name:  Flight  Briefing 

Objective:  To  review  the  flight's  plans,  reinforce  important  points/procedures, 
and  clarify  ambiguities. 

Decisionfsl:  -  What  (if  anything)  In  the  plan  is  unacceptable? 

-  Do  flight  members  understand  actions  (e.g.  formations  and 
check-in 


Data  Required: 

-  Operational  Information 

-  Enemy  Intelligence  Information 

-  Weather  Information 
~  Tasking  Information 

-  Status/Support  Information 

-  Integrated  Flight  Plan 

-  Crew  Material 

Previous  Function:  Flight  Plan  Integration 

Successive  Function:  Ground  Operations 

Products sV  -  Flight  member  understanding  of  the  integrated  plan 

Notes:  The  flight  briefing  is  routinely  scheduled  to  allow  one  hour  for 

ground  operations  prior  to  the  first  takeoff.  It's  end  closes  the 
window  of  time  available  for  crew  planning. 
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2SSl!»«CTKT»i£XT,  TIiOy.  XX'fHHXXV) 

DiMtNSIOfi  KSETfHNSET) 

COHNtW  aSET(MKSfT) 
tCDIVAlEhcr  ffcSETO.CSETd)) 

WiSET-liiSET 
HCRDfi-S 
HPRXT-6 
liTAPE-7 
CALL  SLA« 

STOP 

C 

E^g} 


SUSftOUTIhE  EVEhTM) 

IKCLUCt  'PAilAH  IKC 

C0>#«»i'SC0H:.'ArRI8(HftTRB;.  OO(HEQT).  0£)t(MECT;.  DTIiOi.  II 
IKSTQP.HCLSR  .  liCfiDC.  liPRJeT.  WifiiJK.  WiSET.  fi ■  APE .  SS(HtCT). 
2SSl.'H£G?^Tli£XT.  TtiOk.  U(»tUA%) 

GC  TO  (1. 2. 3). I 

***  DtFIfit  E  veil  I  CODE  I  AS  SUSROUTIilt  ARVL’ 

1  CALL  AfiVl 
RE  TURN 

C 

C  ***  Dtr:Kt  EVENT  COOE  j  AS  SUefiOU^INE  ENOPLAS 

( 

2  '"All, 

SETUfiN 

r" 

L  •**  DEFINE  EVENT  CODE  3  AS  SUBROOTIN?  ENOINTEG 

3  CAll  rNOINTEG 
RETURN 

[NO 


MFA. 


.  MfA. 


n.  i 


c 

Sl«fiOUTIPit£  A^Vl 
IMCLUtJC  PA8AH  :h: 

r 

CO»#IO*i'SCOMi/ATRI8(«snieK  OOfMfQT).  DO(.(WfcOT).  DTMm.  !I.  MfA. 
IHSTOP.KCLNfi.  HCROR.  HPRJ4T.  iihHUh.  NI*S£T,  NTAPE.  SS(H£CT). 
2S5lfMEQT).TM£XT.  T«)i.  XX(HHXXV) 

I GU  t VAICKCE  ( ATR 18 ( 3 ; . PPT :M£  ) .  ( ATK  lei « ) . CXJPPL A>i ) 

EO«JIVAi.EW:£  (ATRIB(5KtXJRIMTtG5.(ATRIB(6).STARTK{ATSIB(7KS£SSl 
£CUlVAi.EH£E  ( ATK IB(fe ) .TLOSS) .  f  ATRIBf  9) . SLACK  J .  (ATRIB(  10)  .AfiORTIKC 
EQillVALEKCE  (ATSiIB{ll).RES).(ATRIB{2).STEPTIH£) 

C 

C  —  CHECK  TO  SEE  IF  RESOURCES  AVAILAfllE.  If  KOT.  FILE  ENTin  IN 
C  •**  APPRECIATE  FILE  TO  WAIT  A#iD  RETURN 
C 

ir{KAii).tt.O.O)  U}  »0  iO 

START-C 

l-SESS 

CALL  riLEMd.AIRIB) 

RETURN 

C 

C  ***  IF  RESEXmlES  are  A.AILABIE.  CALCULATE  HHE  LEFT.  SET  START  TtHt 
C  **•  ECUAL  TO  TNC^.  SET  PLAI«NIN£  INCICATOfi  AND  CHECK  TO  SEE  IF  TWO  CAN 
C  ***  BE  ALLOCATED 
C 

1C  TlEPT.stepTIHE  TNOW 
STAPT-TNOfe 

!F{Xa|3).LT.2  0)  GO  TO  30 
C 

C  IF  TWO  CAN  BE  allocated.  ALLOCATE  Z  SES: JRCES 
C 

RES-E  C 

ii( 3).ix(3‘  -2  0 

r 

C  ***  CHtr>  TO  SEE  !F  full  planning  MMt  15  AVAILAfiLE  WHEN 
r  V**  nc  RESOL’RCES  ARE  USED 

r 

If^  flEFT  LrFOTi»ft)  GO  TO  20 

c 

C  *•*  !T  'FS.  calculate  HIGHT  PLANNING  AND  INTEGRATION  TINE, 
r  SET  ABORT  indicator  tq  NO  ABORT.  SET  LOST  PLANNING  dNE 
C  *•*  TO  ZERO.  CALCULATE  F..  VIK  TINE.  AND  SCHEDULE  ARRIVAL  AT 
C  ***  EVENT  2  {END  CF  PLAN-MN,G)  A^  TN0N*DURPLAN 

DURPIAN-  75*FPTINt 
OUfilNTEG-  Z5*FP"IN£ 

A8OR^INO-0 
TLOSS-0  0 

SI ACK-IlEFT-FPTINE 

CAii  SCHOl;Z.OU«PLAN. ARRIS' 

RETURN 


1)  ' 


c 

c 

c 

c 


c 

c 

c 


c 

c 

c 

c 

r 

C 

C 

C 

r 

C 


r 


*•*  s-’' ,  A.a:.A_sa  ';m£  st'iicrs  Pl-akn;?*:: 

*•*  IHlfG^AnO#*.  S£'f  ASORT  IJCICATOS  TQ  Ac4JlRT.  CAlCUtATE 
***  (OST  PlAWifNil  tike,  set  SLACK  TO  ZERO,  4JiD  SCHEWJLE 
*“  ARAiVAi.  A1  EVEHT  Z  AT  "•RO«*OtlR?LAh. 

25  £XS<PlA)i-(  75-TlEFT} 

E)Uk;NTEG-(  .25*TL£fTi 

asortino-i 

TlOSS-fPTtift  TLEFT 
SLACX-G.O 

CALL  SOCl(2.0URPLAIi.ATRIB) 

RETUiM 

IF  TWO  RESOi^CES  ARE  NOT  AVAILABLE.  ALLOCATE  ONLY  ONE 

30  R£S-i.O 

>Ui<3)»AJl(3)-I.O 

***  QtfCi  TO  Vt  IF  FULL  PLAWiINC  T|»«  IS  AvAlLA&lE  lii^K 
•**  ONE  RESOURCE  IS  USED 

IF'TL£FT.lT.(2.C*(.75‘FFTIH£)«.25*fPTlH£)J  GO  TO  AC 

IF  TIHE  IS  AVAILAfiLE.  CALCULATE  FLIGHT  PLANNINC. 
INTEGRATION  TIME.  AND  SLAa  T'Ht. 

SET  ABORT  INOICATOF.  TC  NC  ABORT.  SET  LOST  PlAWiING  TI»^ 
**•  ZERO  AiC  SCHEDULE  ARRIVAL  TO  EVENT  2  AT  TNOW*OORPlAN. 

0URPIAN-.75*FPTI.ME*2.C 

CXAilNTEG-  25*fPTI« 

ABORTINO-0 

TlOSS-0.0 

SLAC'^-TlfFT-(2  0*FPTIMI£*  75* .  25*FPTI«f  ) 

LALl  SCHOLl2!OUifiPLAN.A7i,;B! 

RETu«>, 


TO 


C  ***  IF  NO.  DIVIDE  AVAIL^^LE  ^I«£  BETWEEN  PLANNING  ANC 
C  •**  integration,  set  abort  indicator  to  ABOR^  CALCULATE  :n5* 
C  *•*  PLANNIW  T{Ke.  SET  SLACK  ECUAL  tq  ZERO.  AND 
C  *•*  SCHEDULE  ARF^IVAl  AT  EVEN'  2  A7  TNOw*OuRPLAN 
C 

AC  IX«<PlAN*.857«fLEFT 
DUfilN'^EG-.  143*TlEFT 
ASORTINO-! 

TLOSS.FPTINE  [OURINTEG*  5*OtlfiPLAN) 

SUCK-G  C 

call  SC^€L(2.0URPLAN,ATRIB) 

SFTUPN 

[NO 


C 


L) 


SijesT'U* :nf  -Hor.iH 

^  .  t  Vi- 

c 

COfWOK  SC0Wi,4TRIB(HATRBK  OOfHtOT*.  OOifNCGT).  DTIii>.  II. 
IH5T0£*.I^LHR.  VPRST.  VSRUh.  KSStf.  VTA-iJE.  SSIMtQT). 

2SSl(H£CT’i.T>i£xr.  XM»«UAVi 

EGUIVAlE^CE  (A’^IEO’j.'^PTIHt  i,(ATKrBf4i.[X^PlAN; 

EOUIVAIENCE  rATfiIB(5).CKS<[»iTEeK^ATSIB(6).STAfiTI.fATR!Bi7>.SESS! 
ECUiVAiEJ^E  (ATR£B(8>,TL0SS),fAniIBf9),StAC^‘,(ArRIB{10).Aa»’':VD 
EGUIVAiEKCE  ( ATfiI8( 1 1). RES). (ATRIBi 2). STEPTIME) 

C 

C  ***  GET  SESSION  MiJHBER  AMO  DETERMIht  HO«  MAM>  RESOWCES  ARE 
C  ***  BEIfiu  USED  BY  ARRIVING  ENTIT> 

C 

I-SESS 

{RCS.GT.I-O)  GC  TO  IC 
C 

C  **•  IF  ONE  BEING  USED.  AND  CONTINUE  TO  INTEGRATION 
C 

CAil  SCHCL:2.0U«IINTEG.ATRrE; 

RETURN 

C 

C  •**  IF  TNO  BEING  USED.  CONHIiUE  CURRENT  ENTITY  TC  INTEGRATION 
C  ”*  with  one  RESdSCE 

c 

10  RES-1  0 

GAEL  SCW)L(3.IXjRIhTEG.ATRIB) 

C 

C  **•  SEAKCN  EVENT  CAi-EMDAR  LOOKING  FOR  ENHTY  UNDERGOING  PLANNING 
C  ***  WITH  ONLY  OWt  RESOURCE  ASSIGNED,  IF  NONE  FX*NO. 

C  **-  GO  TO  50. 

C 

N£XT.»fVE(NtLNfi) 

20  !f  (NEXT,  EC.  O'.  GO*  TO  50 

rALL  COPyf  nE.JT  .NCLPW.ATEie') 

'AypiBi .EC  :.o,  ant;. (pes.eq  i  c:,  g-o  "o  30 
NE  >  ''’•NSUC.k.'  NE  .XT  1 
CO  TC  :c 

r 

r  ***  IF  ENHTv  FOUND  ONLY  USING  ONE  RESOURCE.  REMOVE  ENTITY  FOC 
C  •**  ADJUSTHtNFS 

r 

3C  CA;-L  RMOVE; -NE.xT.NCLNR.ATKIB) 

C  ***  calculate  AWOUN"  or  FIANNINS  time  used,  time  left 
C  **•  steptime  mark  EN’ITv  as  using  two  resources 

r 

used-tncw  start 

TIEFT-STFP^IHE  T^ny 
RES-2  0 

r  rnf.'it  TO  IF  e(,titv  yCULO  -WAVE  WAD  TO  ABOR'^  PLANNING  taR',  v 
AD. (US’  planning  "IMF.  ABC?C’  S’A'US,  AND  .A^-JlA^r  ,Gv  P. ANNIN, 


*  ** 


.  if  EKilTv  iiAS  NOT  &i.feEAD>  !k  AS>yiJ  STATUS.  SPEED  Uf 
:  —  CALCULATE  SLACK  TI>«.  ASD 

:  sc-  ra  4c 

IF f ABORT  I AO  £Q  0}  TFtfA 
DCffiPL  AA-  ( DtKPLM  -  USED ,  • .  S 
TLQS5-0.0 

SLACK-TLEFT  'OORPLAfc-CH^IhrES; 

TO  40 

***  |T  EHTITY  «AS  in  abort  status,  but  mew  SESaSlCE  GETS  IT  BAa 
**•  ON  track.,  set  entity  OUT  Of  ABORT  STATUS.  CJU.CUIATE  SlAD. 

—  AND  !^EE0  UP  PlAAtfilliG 

ELSE 

IFITlEFT  &£.({ .75*FPTJK^  . S*US£0)*( .2S»FPT [MC  U <  THEh 
ASOkTINO-C 
TLOSS-0.0 

0UR!NrEGv25*FPTrH£ 

OURPLAN-  76*fPT!»«  5*USED 
SLACK-TLEFT  { ( .75*F?TIW.  .S-USED}-.  2S*FPTIMt} , 

CO  TO  40 

***  ff  NOT  BACK  m  TRACK.  SPEED  UP  PLAWilNC  A^iO  CALCULATE 
FLAAAiNu  TIME  LOST  yiTH  TkC  RESOURCES  AMO  SET  SLA^K-G 

ELSE 

OiJK.^>!.AM.  I OURPLAM  USED )  • .  5 
TLOSS-FPTIME  {USED*.S*OURPlAN.OURrNT£G) 

SI  ACK-0.0 
GO  TO  4C 
ENOff 
END  If 

reschedule  ENTITV  for  AflCIVA!  AT  [liOPi  an  event  AE^'FO 

***  planning  fine 


<0 

CALL  SCHCL 

RETlffiN 

L 

c 

IF  >«3  ENT  I 

c 

«  *  * 

check  file 

c 

Not. 

r 

FRET  ’HE 

50 

IF (NNQf I ) 

u;  3  ). 

r 

SE  turn 

r 

I'  FNTriE 

r 

*  ♦  « 

ANC  Ai.;:rA 

t  -  ir 

CALL  kWU!  M.ATSSBl 
TLt» IH*  ■  'HJw 
i>T.  c 
re;- 3.0 


***  ihiu  TO  SC?  If  t'h£  riiti  is  if  so'". 

S£’  ABOeJ^  AkD  CALlUlATf  i.OST  Tim  yQ  HE'-  nAWilHS 
**-  it"  Si  ACa  imi  m  ZERO. 

I!  {(  THEN 

AKaCT -Ml.  ' 

[rjS>lA»4»  S57*TLEFT 

tX^lliTEG-  'iA3*fLfrT 

TLGSS-f?TIMt  -  (i-HA^IKTES-  f*0UiPLAA:^ 

SLAik-O . 

GO  TO  eo 

*•*  \1  fuiL  P:W*A:‘-r,  TiNt  IS  AtA3LA£i£.  DlMP'JTt  tLAKAIfiu. 

•**  ikrE&R  TIMt.  AAD  ScALA  Ti«.  S£T  LOSS  TO  Z5R3 

ELSt 

A£aKT!*i£M,o 

DtsLPi«M.7s*??Tme-z.o 

Dukikito- 

TLOSS-C.C 

StAU^TtEPT-n:.C*fPTI«t*.7S)*(  .SS^fPTIHtn 
GO  70  60 
t«3Sf 
tHoIf 


SlHt:CKi:E  EvEKT  :  ,£>0  Of  PLAWIAO,! 

f>I  TLm  N 
f.HC 


soeHi>jTmt  ENOiHTtr, 

PA;47<  !S 

’'OHMOA  scow;  A^C,  lerHl’Se  .  DC'HJC^  .  DPLi>*fnv,  i).  ***  A , 

-kClKR  ,  KvikOG.  .  w^ia,  Wis?’.  ^’a;;  , 

CSSL  .fWf  ’■  .  ^S09.  UiWHi'tV- 

f '■'Civ&lf  HCf  A^c’a  'i ,. le;  .DU'f.AK 

'A'f  :  &■>  ,  :  Arcifc<f%::  .STAi-' V  :.i’CIB.'  ,  M  SS 

f  cc  I*)  Ai,  [HI  f  '  ■■^5:6-5 .  ,5la:‘  .a'pih;;:  .akc^'In 

EOUUAitHrt  ;A..3fII  .PI  i •  o  IP  ^  .  STEP  "  1  Wf 

***  !5[soi>’'j  AV'  ;A’[E-'  Mf'f-fS''  IV^''  i^^rc'-rt ;  j  ■  [ 

.*•  fr  '^  Sr 


i  >  - 


I«SESS 

sni.rC  r  GC  Ta  5 

&r  70  3G 

S  •^TER{i.ATSiB; 

C 

C  ***  S£A«C«  CAiESOAS  LOaj»*G  ^Ofi  tk^jn'  l3«}tRG0IltG  PlAkSI'tG 

C  **-  •Itw  Ohiv  Qhi  C£SuueC£  ASS?GKi.O  IF  h:M  FOMC,  GO  ^0  so 
r 

IG  M£iT-#*i:£(h:LHg;) 

2G  Er.GI  GC  5C 

c;^i  oopv-;..H:>T.w:mii.ATRie! 

It^ffAnilRFlAj  tG,2.0;.A.SD.fR£S.EC-:.";;  GO  "C  3C 
A£  I  T-|»  3yrsrH£XT) 

GO  TC  :o 
r 

C  *•*  U  f.K-:’v  fOLSiO  OKcV  'J5IN4a  xA^  fi£^££.  REMOVE  [Him  fOk 
C  ADA’S'^'Pf'.HTS 
C 

30  CALI  kMOVlI  J»EAT,MLNC.ArEI6) 

C 

C  —  CAL  COLA  FE  AHOUhT  C-  RlAKhlHG  t?h£  yj>£L.  HME  LEFT  Tia  STtPriMt 
C  •**  imr.  £M!^  Ao  USIkG  RESOIIkCES 

f 

US£0-T»i0k  start 
TlEFT.STFPflut  ■  IWb 
&ES-2.0 
C 

C  •*•  CHECK  TO  SET  !F  EHTITV  *OgiC  HAVE  H4£)  TO  ABORT  PLAJAlPiG  tARLV. 

C  ••*  ACCHJS"  PLAKSIAG  ^IMt.  ABOR^  SfATl/S.  kki  CALCULATE  i  OST  P‘.AM*:v,} 
C  •**  tjme 

i  **•  [C  WA'  »iC.T  already  IA  Ag-'y’'  STA'^IS.  SPtEC  ^  R^AASlAn 

r  •*.  .;Aaw.AU  Si^ACir  aMt  AAC 
'  **♦  bO  T  <C 


T  ■  AgS)RTfAC:,£ra’  THfH 
DUTP'lAAWWRFLAS  'J- 'D'*  5 
TlOSS-0,0 

S^Aa.-'TF  '  •:  Oi.VPl,AA-OOP:KTEO; 
C-C  TO  10 


♦  ♦♦ 

r  *■♦■# 


T  EAT''  V<AS  IH  AgyT  5TATt''^.  BU"  AEh  RESOUR, CE  GETS  '■  &A,> 
OH  TAC»  (Aa.'Ji. -"F.  SlAC*  TimE,  SE "  OC-  OF  AgORT 

STATUS  VM)  SPEEC  UR  PS AW^ISG 


^  .  SE 

f  M'l  KT  ,  75.^  pi  if., 

ABy  ’ 


F*UStO'  *•:  25*FPMH{ 


oijr;stg» 


i) 


rN  o  o  r'*  I  *>  o 


is,.  ' i*,'-'" 

t  .  i  .  ■  5-u5tO'-.  ,25*?^' •  i^t ;  ; 

GO  TO  4C 

C  if  S..T  6AC>:  0*1  TkA£»,  SPftO  U?  AkD  CAiCUUTf 

r  **•  PiAW<i:i*0  um  .OST  .ITH  hESOUS^CES  HSZ  set  StAC»-C 

tlSE 

Dl«t.9LAJ<WDt^PLA#i  iJSfDi*.5 
TLOSS-f PT IMt  - ( »US£(J-  . HTESi 
Sim^Q.Q 
GO  TO  4-0 
E»iOIP 
E*ieiF 

•••  kESCFlOt'lE  t»*:ST>  TO  AkPIVE  AT  EKPlAK 


iO  CAEL  SCrtDi  £2.DUfcPLAh.ATRiB) 

HCTi^h 

**•  If  Jw  thTIT>  IS  FOUfcD  Oil  PLAMi«I>iO  ACTIv:T>. 

rwfa  FUtti)  TO  SEE  If  Entities  aae  iiaitiiso  tq  star’ 
**•  nqt. 

•—  FREE  T«t  RESOl^.a 


SC  :f  ,Wsg(  I  )  .EC.Ol  TRth 

,.3 

RE  TURK 

C 

C  If  ERTITIfS  ARE  .AITIlio,  SEMOWf  TH{  FIRST  G*h 
C  •*•  AK0=  Ai.i£>CATE  THE  RESOOF.CE  TC  U 
C 

Else 

'■All  C»A}V!  •  :  .  !  .ATC'E  : 

i  - 

STAR  %TK0i» 


COMPUTE 


iHElk  TP  S£t  ’J  ’he  fcU  FLAW*:iiG  ’JME  IS  A.AiLAEl-. 
I'  R.:’.  SE"^  AfrCK’  A»C  'lA.CUiATf 
***  lost  t;xe  AW)  K*  F.AHKIHG  O'JSATiC^S.  SET  SUlKwC 

:*  HEf-  '  i:  cs'tptihe  ■,  ’u[*< 

[X-'RF-AA.  85’*’l£fT 
Diic:R''EG-  i ‘ ’ 

’! 'OOPISTEG*  S^DUCPlAN' 

sI^A-lH.C  ■ 

CP  6C 

**•  If  f  AitSiSG  time  is  A.AIiAgL*.  P',  ANMRG 

;Ih*>gc  ';m{  a**’'  SlAii  ';«£  Sf’  .OST  ’r  :fs:- 


!.)■ 


PtAAAiNu  'f 


IHt  LtM. 


♦  « 


I  hia-i 

D^PL imi- .  73*f  PT I W £  .  G 
Dl?S5hTEG-.25*f?T!M£ 

TLGSS.C  ; 

SLACK-UtfT-{(Z-0*^PT;»<£-.  • 

C-G  TO  6G 
tNCIF 
E»hD;E 

c 

C  •**  StHiDOlE  EVENT  2  {[«  OF  PlAWiiHGJ 
C 

60  CAIL  SCHDKZ.OUftPLAK.ATREB; 
fiEIUWii 
END 

f 

c 

SU&CDUTlhi  OTPUT 
IVILUDE 
C 

scowl  ATREa(HATRE>.  DOt»ltCT'.  OOLJWECTL  DTKOb.  II.  »tfA, 
lHSTOP,hCL^,  hUiOK.  hPiCHT.  HkHUk,  lOiStT,  NTAPE .  SSiWCCT;. 
2SSLiW£QTKliiiAT.  TiiOt.  XX.'HHJtXV’ 

ECUIVM.ENCE  (ATRI6(3).FPTINtM*rRIBf4i.[XJJiPlAM 
ECUrVAl.£»*Ct  (ATSlSISl.DURIliTEG:. 'ATRIBiej-STAiiT’  .  (ATRIBt  7  .SES5 
ECiilVAtEWlE  (ATRI8(8y,TtOSS)  .fATRIBfSy.SLACX;  .|AT«IE{!0),Aa0C.’:HC 
£OUIVAlE!*C£  {ATRI8(II).R£S).(ATS.I8(2}.STEPT1P€) 

C 

IMHHRUA  GT  n  GO  TO  !C 
ta^ItE(NPRN:.:0)PftOJECT 

ySITEfliPfihT.T  ABfiTr  ,  •  ABST2- .  •  PTl  PT2  .  SU. ) 

-  slx2  .'tlcsti  .  tlost: 

f 

10 

AERr;,rrHjM/5 
- ■ 1-CCAVG; :  ^ 

P^J-'Ll^vGsX 
SL*.  1  "CC  A»G{  3  i 
Si k,2 *CC  AVu*  6  j 
nOSn-:CAVG(2; 

^  ‘  OS  ’  t  ^  A  V  o  ?  ' 

«ilTEft*piiNT.2CiAg#iTi .ABfiv.pn SI TICS’ 1, nos’: 

Sr  TURN 

2C  fORHA’.  1* .2FS.0, lx.6f T  3; 

30  fORKA’' U.6HMOOEI  .Zj.a;?- 
fNC 


D  : 


VC 


GEN  Gj&¥LO*iC  S01»*»“ELL  “^"ta'S  ^5X?C!  '’■■'?2  ’00  NNV'>'^i^ii’2 

2 ’?  5 

mi£  •JN  '^  IS  UiNu'^^S 

EC  J^VAd£NCE.ATW;B*M  tqt  *7^(612,  S^iEPTiMt ■*’7Bje'3',PPT»*.«£.  DUWPLAN 

EGiJVALENCe'A'’^lBi5i  DU<RjN’^G  A^RIg{6i  S‘’'A»r-ATT;iB<T',  ScSS  -■^^a-es  "aOSS 

eQlflVAL£NCE>A75SJe<S>  SUOCfATRJBilSi  ASORTIN&A'TRJBi’’;  NLJMR£&,XAn>  .sCkyA^i, 

EC 'JVALENCE,-ATR*6i  1 2, -GSOO«=  S 

INl  TtAUZE  720  V 

PWORfTVO  LVF(iy2  LVPi'*) 

fNTlC  W(3H3  C  Xj«i>tao  XX42>-505 

"HMST  XAf3»  MSS  AVAiLAe^UTV 

MONITR  trace  «40  0  20C  0  ■'  2  -3  ’  2  j  4  S  5  '  S  S  10  ”  14  15  NNQr> 

NETWCRK 

CREA  CREATE  0  6 
ACttv 
ASM 

ASSIGN  TCT-x.x/1)  GNOOPS-C»iFtiA4(6C  SC  3)  S’’^ ?■"«£» •’‘O’  GNOCPS  PPTiM£._»,rcy.i. 
10.213-Zt  S£SS*1  XXf’i»xx(i>» 'C 
ACnVTV.i  10  MASS8RE 
evil  EV'ENT  1  1 

ACTIV'TV 

TWMI  TERMiSA'Tc 
ENT'  enter  1  1 

ACTA'!'’> 

COL’  COLCTdl  INT;6i  FPtRTlME’ 2&5V5 
acttvitv  AA 

AA  GOON  1 

ACTTVirv  AgORTlNC  EC  ’ 

AC’'’V'”v  COL  3 

COl2  COtCi'T?.  ■^LCSS  LC5’  ^mE  '  205  O^S  0 
acttv'.tv 

G2  GCCN 

AC‘7'V*>  2  SLAC^  ee 

COL1  COlCT(3',  SLAC>‘  SLAC*^  ”M£  '  20  5  C  5  C 
A-CTTVT’V  G2 

BS  GOON 

ACTTV'^vO  GNOOPS-CIVPRM-,  VO  ’9C  4i  GNOOPMiSSiON 
ASM 

ASSIGN  GNDOP3-UN5:BM(eC  SC  3'  S'’TPT‘ME-TCT-GNOCeS  fptimE«  JN^RM 

'0  20  2:  SeSS»2  *K-2i-xXi2  -’GOJ«P!_aN-GOl,'OiNTEO='0  ’'cOSS-0  SlACi- *0 

ABO«TiNO»0  NTJMRES*© 

AC  vyiTv 
EV'2  EVENT  ' 

■  RM2  te«m  ‘AATE 


n  I 


"  -  6 


EN"'2 


:.Z^i  ZZ^Z'  4  . 

ACTTV^Tf  .  ZC 


CC  GOON  1 

AC'nvtTv  AaorJTiKi;-  eg  ’ 

ACTTVITV  COtfi 

COLS  CCLCT(5i  tlOSS-LCST  TIME  220£a'SC 

ACTTVTV 

ZAA8  TEfObtiK^LTE. 

CCHJ  COLCTf6»  SLACK  SLACK  TIME  £V5  0 
ACTfvrrr  ..ZAAB 
ENC 


1)  '  i 


3^^  ~0*' 

--jc  A 


GEN  GAVLO«y5^SOVV^.U-  Ts-SSiS  15X25  7-1Z'19*2  100  N  N  v-V  N  N  72 
UMi^S?  138 
TIME  UNJT  IS  MiVJ'icS 

ECU»VAi£NCE.-A7Bl8<n  TCT/ATRiaiZ)  ETEPmi£  ATWj&iJ,  EPmiE  ATUfa**”  OURPlAN 

ECmVALENCE-A'^lStSi  OUNiNTEG  ATW»B<6!  S7ART-ATR-8tTV  SESS^ATWiBlBt TLOSS 

ECUVALENCEfA'nPiewS#  S4>CKyATHiei*0;.AE<»TI8ayATRlB<11),NUM«EaiO.;3j  RAUAiL 

EQ18VAL£NCEATRI8(  1 Z;  GNDOPS 

ffsPTIAiJZE.  T2S  Y 

PRIORfTY/1  .LVP(  1 W  LVTM  j 

1*faCXXp>-3  0  XX(1i-180  XX(2)-505 

THAST  XXP)  MSS  AVAJLAiit'LrrY 

MONTW  TKACE.140CJOO  O  1 -2.  3J2L3.4.5.6.7.J3.9,!0.11,U  15.IVtfiCt11 
NETWCWK 

CR£A  CREATE  D  .6 
ACTTViTV 
ASM 

ASSKJK  TOT-XXll)  GNOOPS<N_-NFRM<e0.8a.3l.STEP'nM£-TOT^NDCPS  FPTWi^E-UNERMi 
15J»^  SESS-1  XX;l>-XX(n*tC 
ACTTViTV,  1  30  MASSBRF 
EV11  EVENT  1  1 
ACTIV1TV 

TRM1  TcRMiNATE 

ENT1  ENTER.  M 
ACTIVJTV 

COL1  C<X.CT{1HNT(«)FPiNTHUE’20V5 
ACTTVITV  AA 

AA  GOON  1 

ACTIV;'^  ,AB-ORT1NO  EC  1 
AC'HVI’V  CaL3 

COl2  COLCTf2i  TIOSS  Las'"  TTME  '  2C.5  SS  0 

ACTTViTV 
G2  GOON 

ACTTVlTVO  SLACK  BB  STUOv 
CaL3  COLOT(3i  SLACK  SLACK  TIME  ’  ZCS  2  5  2 
ACTTVITV  G2 

BB  GOON 

ACTTVITV  3  ONOGPS*UNFRM(irC  ’90  4;  GNOOPMlSS.'ON 
ASN3 

ASSIGN  TOT.XXP;  GNOCPS-ONFKMieW  80  3»  STEPTIME-tqt.gnOQPS  ppttmE-lINPBW? 
’5  25  2i  SESS*2  YX(2-»XX<2: •  n.-  CMjPPljXN»0  0U«!NTEG-G  TlOSS«C  5lACk-0 
A80«TTND*0  N(JM»ES-0 
ACTTVITV 
evi2  EVENT  ’ 

aCTTVTTV 

TBM2  terminate 


D  ' 


EN  ’^2  CH-^rq  2 

CQi.A  COLCT(4..iKT46^_FPKT'»,«i2^aS  C-5ii_ 

AcmnT>...cc 

CC  COOK  t 

ACTIvmr.jSBOSTIKC  EC  1 
ACTTVTTV  cots 

COLS  CCH.CTt5).rL.OSS.LOST  2-2a^  e 
ACTtVJTV 

2AAB  TERMlKtATIi 

COtfi  COCCTC6)  SLACK  SLACK  TUylE  2,256  OS  Q 
ACnVJTf  ,ZAAB 
EM3 


D-lf 


MOOfu.  ? 

GEK  GAYL0<^0_S0WELL.'nrteSiS_2GX3C  7"iZl6fi2  TQC  M  >  N  72 
LtM-TS2  11  9 
mi£  U»vT  IS  M»#YUTE3 

eGU!VALEW:t'ATSJ}e{i(  TOT^XTRiSOi  STtPriMt/ATRiBili  PPTMUfc  ATR^B-4'  DURPU^N 

EC'J1VAi£^^CE/ATR<a^5),OU«^K'nEG.ATR(B{6^  STA»T.'ATRt0iT)  SESS  Ar«ia*6>  'aOSS 

ECUIVA1£NC£;AT«iB<B)  S4-ACAMTRie(i0).A8ORT>»>iOlATR3iS4t1i  ?«UMR££.>jr^3>  RAVAJi. 

ECUVAi£>K:E/ATRJB{12).GASDGPa 

INITIALIZE,  720  V 

PafOWrT«T.LVF{1^2  LVFin 

INnC  AXOl-3  0XXflHia0XA(2}-5C5 

TKOST  Xxm  MSS  AVAlLASILmr 

MONTR  trace.  ! 40  0,200  0.  1 . -2.  3. 1 2.3.4.5.fi.^.a.9. 1 0. 5 1 . 1 4 . 1 5  hiHQi  I ) 

NETWORK 

CREA  CR£ATt.0...6 

M-w\j 

ASN1 

ASSIGN  TOT-XXi1>  GNOOPS»yNr«iMl<60  a0.3LSTE.P'nM£-TOT-GNI>2PS  FPITMc-UNFRil, 
2C  302)  SESS«1  JCA(l)»XA<1>»:0 
ACTIVlTV/i  30.  AAASS^if 
EV11  EVENT  1  1 
ACTIVITV 

TTiUt  TERMINATE 

ENT’  ENTER  1  1 
ACTIVI  r> 

COL1  COLCT(iilNTr9).FPLNTlM£*2Q^S 

ACTIVITY  AA 

AA  GOON  1 

ACTTVJTV  jABORTINOEG  1 

ACTIVITY  cot3 

COL2  COLCTf2;  TLOSS  LOST  TIME  ’  20S  O'?  C 
Acnvrrv 
G2  GOON 

ACTTV1TV2  SLAC*i  BB  ST\X>v 
C0L3  COLCT(3!  SLACK.  Su4CK  TIME  ’  --GS  GS  C 
ACTTVITV  G2 

BB  GOON 

ACTrV1-^y,-3  GNOOPS'UNTRMlirc  190  4)  GNOOPMiSSlON 
^N3 

ASS!GN  TOT»XA(2l  GNOOPS^UNTRMlBiO  90  3)  STEPTIME “TQTjGNOOPS  Fot>mE*‘>^fpm,|' 
20  30  2)  SESS-2  XX(2««X,XC2;*  1C  DO»PL>N*0  DG-RlNTEG-e  TIGSS-0  SlAO  -0^ 
ABORTIno-0  numres-o 

ACTIv'T-y 

EVt2  EVENT  1 
ACT’V?"^ 

TRM2  terminate 


I) 


E*j’'2  ;■ 

AC*  ■  V'  '’■ 

i-Oi.*  CCk.C'^i^r  ^ ^UT-kdE2S.2'i  0 

ACWlTi  ,CC 

CC  GOCK  1 

ACTrvrnr'  Ag^DSTtNC  cu  ^ 

AC^TVl'P''  COL6 

COt.5  COI.CT<5i  TLOSS  LOS^  r!M£  2^-5  15  £1 
actt/itv 

ZAAB  TEf^NATE 

c<Xj6  colct(6i  slack  Slack  mi€  2.205  05  c 
ACTIVlTr  2AAB 
END- 

KiK 


D  ;S 


vcz-f.  : 

G£K  GAVLOWD  SOWELL.Th£SsS_2SX35  7-\Z  ^9i2  ’00  S  “<  t  ■>  ‘i  N  72 
UU>rz2  ^3,8 
m4£  L»^T  IS  Mi^hJTES 

ECJtvA*.FM:e/ATR»»<n  ToT/ATWiao)  sTEPmie/A'rejeot  ^ptbaeatsub^'  DWPLfcXi 
ECtBVAL£«WCe./ATPlB(5i,OLWiNTtG-A‘mi8f€,i  STAPTiATRlBiTi  StSS- TLCSS 
EQUIVAUEM:E/AT*?JB(Sj  SLACiCATnjRtiO)  ABO^ni^ATHieCt  ‘1  fiUMRSSXxas  ravail 
EGU«VALEN«:£/ATRiB5 12*  GNDOPS 
lAaTlAUZ£..720  V 
P«lO«iTY/1  .LVF(l  K2  LVP>  1  ^ 

INUC  XJfO)-3  0  XXn>»t8C  XXC2)-5«tt 
TIMST  KX(“3;  WSS  A„'AitAo*UTV 

TRACE. 1AC  OiOOC  -1  -2  3  34  5,6  >'  i  8  <0  11  14  15  (SiM3i1) 

KETWORn 

CHEA  CREATE  C  6 
ACTTV1TV 
ASM 

AS&GAi  TOT-)U,1i  GAOOPS<MJA<FRM<6C  SO  3i  STE.PT>ME-TOT  GAOOPS  FPT!M£«iJNFRM, 
:»  35  2i  SESS-i  XA<l*-K>,n*iO 
ACTTVnTV/l  3C  MASSHR.^ 

EVIt  EVERT  I  1 
activity 

TRM1  TERMlAiATE 

ENT1  EATER. 1  1 
ACTTVirV 

COL’  COLCT(1l  »AT<€i  FPLATtME’  itVS? 

ACrrjITY  AA 

AA  GOON  1 

ACTTVSTv  AaORTIAD  EG  t 
ACTA.'I'^  CC.L3 

COL2  COLCTiG,  'LGSS  LCST  ’'■.ME  1  iO.'S  CS  C 
ACTIVITY' 

G2  GOOA 

ACTtv.'^’G  SLACn  68  S^'JO'^ 

COL3  COlCT(3j  SLACK.  SLACK  T|M6  ’  213-S  OS  0 
ACTIV'ITY  G2 

ee  GOOA 

ACTIVITY  3  GADCr'S*L'A*=^RMO  '0  180  4i  GAOO*=**.1lSSlON 
ASA3 

ASSIGN  TOT.xXut  GNOQPS-OAFRMrtW  *0  2i  STEPTiME»TCT.Gf>ii»ps  pot?me-UAFRM-. 
2S  35  2)  SESS*2  X)fQi«XA,2-1G  !X»«PL>N*0  DJPINTEG-C  Ti.cSS-3  S^CK-Q 
ABO«T!A£>*0  N’JMRES-O 

ACTIV!'^ 

EVT2  EVENT  I 

TfiM2  TEBMINATE 


CGU  CCi.CT<4j  !NTf6).FPf«mi£2  2SJ^  0 
ACTTVITV  CC 

CC  GOO^  t 

ACTIVTV  >eoR’^*KO  £C  * 

ACTrVITV  COLS 

COl5  C0LCT(5;.TL0SS  LOS"^  T1M£  a-5  C 

ACTIV!TV 

ZAA6  TE<«MiNATi: 

C<X6  COiCTfB)  SLACK.SLACM  riaM£  2,20*5  05  0 
ACTIVITY  .ZAA8 
CNC 


3'^^M  _  9'"^  Z-'Xi€ 

‘.*32£.  Z 


GEK  G/i'^LO«iD_S-DVl»'EU.  THESiS  30*40  Ml  *592  ’iX  \  w  N  N  ^2 

LfM^T'5  2  13  8 

TWUt  tlN^T  «S  M*NLr>eS 

EG m VALENCE  AT??>eiV)  TCT'ATRiBai  STEPTlMEATWiBiOr  FPTHWE  ATRiB-A^  OimPLAH 

EQUiVALEMZE  ATRiejSKDURiKTEG  ATfOB^ei  STAPT/ATRsBiTi  SESS<ATR»6<8i  TLOSS 

EQmVALENCE-'ft’nRIBfS/  SLAClOATRmilQ'/JkaOHTihUiATRjBil  RAVAiL 

EGl»VALE»^EArRie<  12)  G-SDOPS 

IA0T1AUZE.  720  V 

PRKJRiTV^I  LVFOVl 

)N71C  X*it3l«3  Q  iUt<1H1«QJO({2)-SC!5. 

TIMST  XX(2(  MSS  AUAiLAStUTY 

MONTR  TRACE  140  0200  0  -1  2  3  12  3  4  5.3  71  S  10  11  14  15  SJil'ii 
•^TVWOfti\ 

CHEA  CREATE  0  ,,e 
AcnvtTv 
ASN1 

ASSIGN  TOT«AA(1»  GN£X>PS-UNFRM460  30  3)  STEPTIME-TOT  GN£X)PS  FPTIME-ONf  RMi 
30  40  2>  SESS-1  XX(^>*XX(1>*iC 
ACTTVITV/i  30  .MASS8RF 
£W  EVEN''^  1  1 
ACTtVITV 

TRM1  TERMi.ViTE 

ENTi  enter.  11 
ACTIVm 

COti  COLCT(1HNT{6j.FPLNT«^1.2(VS5 
ACTtV1i>  AA 

AA  GOON  1 

ACTIVITV  ABORTINC  eg  1 
ACT»\nT>  COi.3 

COl2  COlCTCZ,  'LOSS  LOS'  ^'ME  ’  2aS  GS  0 
ACT'VT^ 

G2  GOON 

ACT’VTVl  SLACK  88  STUDv 
COLS  COLCTfj;  SLACK  SLAC.K  'LME  ’  2CS  CW  H 
Acnvirv  g2 

06  G'OON 

AC'  Vl’>  3  GNDOPS*L'NFRAI;irO  150  4;  GSOORM^SSiCN 
ASN3 

ASSIGN  tot-*,A(2)  GNOORS-UNFRM(60  80  3:  STEPT'ME-TOT  GNOOPS  FPTlME-oNFRM. 
.30  40  2)  SESS-2  )‘.*2)-XA(2r'’0  OJRPLAN-0  DO«iNTEG-G  tlO55»0  SLACK. 0 
A80RT1^£>-0  NOWRES-O 
AC  r^\)'r\ 

EV12  EVENT  1 

•^RM;  terms  NATE 


t)  -'i 


>--wTi  v:  T  j 

CCK.4  C.Oi.CT(4,  JNTfB/.FPKmMei^lSaSO 
ACTTVITV  CC 

CC  GOOK  1 

ACnV’Tv  AflOFTINi;  EC  ' 

ACTIVITV  COLB 

COLS  C0LCT(5).TL0SS.L0ST  TiWe  2  iG-S  0 
ACT1V>TV 

ZAAfi  TIEnUlNATE 

COLB  COLCTffij  SLfcCK.SLACK  TIUE  2  205  (VS  0 
ACTIVITY  .JIAAB 
ENC- 


f> :: 


MwLtk.  h 


GEN  GMTLOfiC..SOAE^  35X*5  '  '2..?W2  %  \  t.>  *.  \  " 

LiMiTGJjrje 

T1MC  USi-  iS  MiKU'-'  ES 

eCL-iVA‘XV«:E»A'W|.8<^.  ''■C^^ATR-BiZ)  STEP'TlMt  ATRiBiJ-  Fi'TWAEu-N.o,*;  Cv^KFiJS^N 
£  ^ t  .f\,'  Ai,£.  ftiC  £.•  A  TWi  9<  5  j  Pl^jNTEG .■  A  7 A  <6<4  S  AS  t  .“A  TR  s  6 1'  r-  ,*£  A  TRiE  j  B »  T  l  OS  i 
t;-U.yAL£NCtj!'A’RI8<S,  SiACivAtfUfei^O)  APOfiTmO-ATRsBt*  1)  VUMRESf'XXi3'  RAVAI' 
EQU»VALENC£/ATRiei  ;2j  GKDOPS 

IM7IAUZE  720  V 
PRrOftJ  TV  1  xVF  i  1  L VF  t ) 

‘NTLC-X>  (3.»3  P  XA|1>-160 

M,'i5»ASS  AVAiLAE.Ly'l  .' 

MONTw trace  iiao2oo:  -i  -2  .3  ^2345.6^  as  *-  u  t'-siNC'’: 
wenvczj-, , 

CR£A  GfiE;.''TEC  6 
ACT»V!rv 
ASKt 

ASS40K  roT-^X.  Vi  GNDORS-UKfSAAlM  SC  3i  STEi»'1ME.TCT<ii.,iX)PS  rPTA4E*UNrCA»( 
35*5  2  5£SS"l  Xx  l>«*x  i)-»o 
ACTlVfPr  ;  30  MASSDRf 
tvEN*  1  : 

ACTiVlTY 

TOW1  TERWiiS^rc 


fNTi  ENTEO  1  1 
ACTTV';'^ 

COt’  COC*^T(l!  iRTjft,  2tVS5 

ACTT/Tv  AA 


AA  GOON  * 

ACTV'^  ABOR'^’NOEC 
ACT:v'~>  20l3 

CO^C"  7-  t;^0SS  .OS^ 
/  C 

g:  gcos 


•"*ME 


*c'''v"T» ;  sui  >  ee  2TX''^ 

.,..'1.3  COlCT  3:  GiAuK  7’.v£  *  27  t  3  ^  3 

AG'^'Iv^"*  G2 


ee  Gcos 

ACT'.V'''"^  3  '.jNCOPS-'^AA'PMi!  .'c  ^SC  *■  GSOOPM-SS'GN 
ASS3 

A  j 5<GN  TC  '  »  <  »  i7.i  GN2X2P S«\j'N'^RM*60  SC  3 !  5T£P’'-A4£'«T0  i  GKC*OP7  ‘■P 'lAif  ■  .?4S^SSX*- 

35  45  s-'i  S£SS«;  i»'2't>'Xxr2'-**C’  Ds7«Pu** '  *■ 

A60F  'A'O-0  S'L'MetS»-7 

*r 

fcV^2  EVES’-^  ‘ 

AT  T!,^,:— , 

TCfv;  ■^EBMiNA'^E 


COL*  COLCt{4,  i.SrfS;  2Q!^  C'5  0 

&CTT\nT>-  “C 

CC  GOOK  ^ 

activity  AfiORTiH^tC  ' 

ACnviTY  COL6 

COLS  CGlCT{5<  TLCSS  LOST  lUAE  3r5  0 

ACTTVITV 

ZAAi)  TtJ«4J»LATt 

coLfi  ccH.CT{i6>  SLACK  Slack  TttAE  2-*;as  as  0 
ACTIVITY  ,ZAAB 
E-NO 


f)  N 


MCOci.  ' 

GtN  GAvlC=^0  SOVVEU.  '"‘^.E:S_4ax5C  r  N  S  »  ^  %  N  '2 

T»«£  UVT  *5  MiSU'^tS 

EQLffV'Ai.EKCE-A'^iBnt  ‘C^-ATKiBii:  5TEPTiM£-if  ^iB'3=  PPT»ME'ATS}iB,4.  C>U«P>-AN 
eCU<VAiJENCE'A'nRiS«5i  DURUHiTEG  ATRieia*  ST&PT/ATRta-  "<.SESS-ATRlBiS;  TiOSS 
EasJVA££M:&ATyjjB4S;  Si>CK?Ame{10)  ABOP':Si>'ATR»B(t1)  NUMP£S>X!;‘:  RAVA:i 
eCUVAt,£KC£;ATRi8u;;  GKDOPS 
‘MnJMJZE  75:0  > 

PRlORfTV'- f  LVEO  LVE ■  •* » 

♦NTLCJ>Ji(3>-3  0  AA^Ii-iaOXXa^-SOS 
TIMST  XjisUs  MSS  AVAiLABiUTV 

MOt^TR  trace  -40  0  200  0  -1  2  -3  1  2  3  4  5  6  ?  6  i  10  ii  U  16  KLNW.H 
Artwork 

CREA  CREATE  0  6 

ACTTV'^tv 

ASKl 

ASSIGN  TOT-Xa*1)  GAOOPS-USFRfckWJ  83  3i  STEPTTME-TG  '  GNOOPS  FPTiME*- JSrRM, 
40  50  2*  SESS^i  XX(iH*,A<l)-iC 
activity K  MASSBRF 
E'vH  EVENT  1  1 
ACTiVIT> 

TRM1  TERM'KATE 

ENTi  enter  1  1 
ACTIV."^ 

C<Xi  C<XCT(iMN‘^i5>  s^PUSiTlME' 20^*6 
Ac  nv  T>  aa 


AA  GOON  1 

ACTtV."^  AEC  r,?OEC  1 

ACTi'vt^v  Cl.  . 

COv_r  COlCT.;'  ’^OSSlOS'  T.WE  •  206  0“5Q 

ACTIV"-’. 

g:  goC'S 

\cTiv  2  SijiC**  ee  STUDY 

COl3  CClCTO;  Si>C!>  S-UaC*"  t,m£  *  2C6  0  5  G 

activity  g: 


SB  GCX?N 

ACT!V>’'r  3  GNDOPS*  JNFBMiiTO  1V0  4!  GNOCPM  SSiON 
ASS3 

ASSIGN  ’CT«X>.2,  GNOOP5-UNFPMt6C  60  3;  S ‘^EP'' ‘ME  »  TQT -GliO^PS  ft  t*ME-UNPPA* 
40  60  2:  SESS-2  m,2>«*  10  DuBPi-AN»0  Dt  P:NTeg»<'  "^uOSS-C  SvAC/"-*:- 

ABOPT.^O-*:)  NUMR£S»C' 

activ;^* 

EV''2  f.VEN^  ‘ 

Ar^iV'^Y 


E*.'’2  EN’EP  : 
acT!v;t» 

COU  COLCTif4).l>iT(6j.‘^f»f^T!M£2-2a'5  a-5  0 
ACT;V"Tv  cC 

CC  GOOK  V 

ACnvlTv.jiaOFJTfNCEa  t 
ACTTVJTV  COiJS 

COLS  C4XCT(5j  TLOSS  ICST  TmA€  22m  m  0 
ACTIVITV 

ZAAB  TEratliNiATE 

COLfi  COLCTI6!  SLACK  SI/jCI'.  TawK  2^'S  OrS  Q 

activity. 

END 


MOOti.  G 


Gc^  Gj^yLO«iD_SOvVE'LL  TH£StS  i5X5i5  ^DC  N  H  v  >  N  ’2 

LIMITS  2  t3» 

TTIyll£  U»i»jT  IS  MJ>^UT£S 

eCufVALE^CE’A'miBlM  TOT(A'rR*Bi2)  5TEPTTMt'ATKi6t3*  FPTTME  AT*;»e<4f  DOfiPLAK 

EQUIVALEAiC£/ATi?iB<5»  DlWnKTEG  ATH*©46i  STAfiT.'ATS»{S<T»  5ESS'ArRiBi5!  "H-OSS 

EaiJiVAi.£NC£;ATRi8{S)  SLACiOATWe^lOjAflOR'n^iO-ATKiBtlt)  »^JMH£&XX,3j  RAVAiL 
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Captjun  n»mis  I  Oi\lord  uis  bom  on  ^  F«bru«rs  .n  Lal»\ene. 

indiaast.  He  grailuilcd  from  Hsnug.’  Hsi^h  Scnooi  in  Conyefi,  Geofgii.  in  iro 
ftnoHleii  Cisllese  m  Eiston.  P«;rmv/ivviia.  gradiuling  lAith  •  Ba4:helor  of 

Science  in  MeclumicAi  Sngineenng  in  Ma>  HiiU  L'pmi  graduauon.  he  receded  i 
reserve  ennunisron  ir  die  USAE  and  served  has  fim  tour  of  duty  al  nghl-PaKerson 
AFB.  Ohio  He  began  mu  an  Acquisition  Project  Officer  for  Acroaaiaical  Svsiem 
Divisions  Deputy  tor  Reco.maisance.  Stnke  and  Electronic  Uariarc  Sv stems.  Ptcciuon 
L  PC  anon  and  Stnke  System  (PLSS*  System  Program  Office  iSPO)  tASD'K'kO*'! 
where  he  managed  the  deveiopmeni  and  test  of  the  communicanofl  hardware  and  PLSS 
Inteliigeoce  Augmentation  Sv&tem  Lpon  compietion  ot  hiv  tour  he  was  rea>signed  lo 
Headquarters  TacttcaJ  Air  Command.  Deputv  Chief  ot  Staff  o>t  Requirements.  Langlcs 
AEH,  \irgtnia.  where  he  served  as  a  Tacticai  Sysicnu  Acquurtrun  Program  Manager 
tor  the  Director  of  Inter eper ability  and  intcgiation  There  he  was  responsible  for 
developing  and  drtenuiiig  the  Tactical  Air  forces  iTAMi  Mitaion  Support  System 
I  MSS)  budget  and  program  submittals  and  managed  the  software  deveiopmen*  ar.d 
integralion  contracts  tor  the  TCf  MSS  II  (automated  mission  planning  yvstem) 

I>unng  the  period  ot  December  1*^'^','  March  i***-!!  ('aptaun  Cravlprd  iwuc  leo  ;  eid 
teams  to  Southwevt  Asia  during  Devert  Shield I>eserT  Storm  to  insiaJI  MSS  II  upgradei. 
and  tram  comba;  aircrews  on  how  to  mission  plan  with  the  modified  MSS  II  DS 
I  pon  return  to  I.angiev  AhB  (  aptam  Oavlord  received  a  reguia;  commission  and 
assi»ted  with  the  requirements  development  for  the  nest  generation  Ait  force 
automated  mission  planning  sv  stems  until  entering  the  School  of  Sv  stems  and 
I  ogistirc  Air  force  Instituit  of  Tevhnologv.  m  <lav  N'^1 

Permanent  \ddrevs  i  22**  C  or  lev  Rj 
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Ciptiun  Tieot^c  1.  on  .<  Apn?  !^.0  ;r;  Haj^rrsistlJ  <  -jii;- 

H£  graduated  irotn  Jacksooville  Lai>oraU)r\  High  SeK^>t  in  JjK.i.u'mvtik.  Alahaou.  m 
IMSU  juid  aOcoded  tb«  L'aivmsU  ^  G'^orpa  Irotn  IStetJ  to  IMIid.  earning  a  VSim'  tr 
Bimoesi  Admimauation  degretr  (rvi  V  maiugcmcnl  loiortnation  tvuemsi  in 

Mareb.  IVB4  He  remaioed  at  the  ^  '.aierMts  of  Georgia  punuing  a  Aiaiart' '  rn 
Matiagemeiti  uoul  he  reeci^ed  a  revei-ve  coounmioa  lo  the  USAI  lu  Juoe  oi  and 
was  uline^ueaU>  cmlkd  to  active  dut>  in  Jul>  ot  IVtid  I'pon  ertenng  ac?."  e  iu!>  hr 
\ev-tti  an  art  Amonm  h>s(rfiis  Aiuivst  ai  Headrjuanern  TacliLal  Air  (\muuand  ilAC'.- 
rcsporiMbie  for  devckiptng  and  •  *% ieienj  i omputer  nmources  reiiuimnents  tor  the  f  !»> 
r  i  i  t .  r  4.  Pave  Tack,  and  XLuioa  llanaitig  hvnUnu  tn  !sthf>  he  avwaied  hfti,^r 
re\(H>n\ibitiUe>  av  Advanced  hghter  Programv  Manager  He  managed  comptiter 
roourcei  it!rc>cic  re^uircmcou  and  piaaoing  activities  assixuied  vvith  advanced 
ciauUicd  Taciicai  Air  t  orcev  s> stems  He  rrpersenird  T  Ai  ‘  in  devciupnient  ol  the  iniiiai 
Advanced  1  acticai  Fighter  sofiwarc  supp*»n  cotK'ept  He  wav  resporviNr  tor 
vludv  ot  embedded  computer  vvMcm  vot»uarc  supjion  rrunafieiincnl  In  i  At«>b%‘r  Jiv^T 
rjpuin  'v!>wel!  received  ^  rtg'ila;  commu-vion  In  Ma\  IdKK  he  rcawigncd  »' 
Heaiii^uarter'i  Air  f  ;>rve  Swiem'  < Ommand  •  Af  Si  *  where  he  vened  as  a  Mivvum 
Hannmg  Sv  stem'  *  htlccr  revponviNe  lor  (.xmsolidating  AFS(  acquiMtion  of  ciraiez 
tai rival  airiijt.  and  Special  OpcraJ.i'.»nv  f  orcev  mivvion  planning  vv stems  He  stxfdmaleJ 
doevlfoo  .ind  re"a»uf«.e  vommitments  lur  this  eftort  He  initiated  and  led  developmer.:  r 
■\1  Si  (  (  endiHvd  nuuiniap  which  l•ierUl^led  vhoiltallv  m  capabiiitirv  and  pri>p  w  1 
managemcni  -eaJignment  (  apiain  Sowell  wa'  assigned  is  a  siiidcn?  :  >  the  Ak  I  tf.e 
institute  of  1  fchriotoCT^  Sch<H>i  H  Sy'Ucms  and  lAvgistus.  in  May  IWl 

Permanen'  Address  22 1  Noah  \  iHes  Kl 
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1  Di4  thu  yyxamh  conmbutf  (o  a  otmat  itieimr  prcgca'* 

1.  Yn  ti.  Nc 

2-  Do  you  believe  this  icaeuch  topic  is  tifniftfanr  caoujb  that  u  woul4  tu>>c  been  reseiu'cnc^  c* 
ccotaaedj  by  your  onarmaiion  or  another  agenc)  if  AFIT  had  oat  teseircbetl  u'* 

a.  Vts  b.  No 

j  The  faenefim  of  aFTT  icscamh  can  ofun  be  eapft&acti  by  (he  n^uivatcnt  u’u:  vou.'  iit-:,- 
vtsx:-ei  by  nm«e  of  AITT  periomunj  the  rueahA.  PleaK  cciniaie  >*2u:  tn^s  rucarcr. 

U.e  cost  in  lena*  of  aat^o^'  aadw  OoLan  if  u  hat!  been  aocomptuncP  unOc'  cotrr.c.  r* 
na:  been  done  uHaouse 

Man  Yeart  .  $  . 
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